Lama 改成手动安装，新增JsonRepair

Update PyProject Toml - License

CNAME

@@ -0,0 +1 @@
+mixlabnodes.com

README.md

@@ -1,15 +1,27 @@
 ![](https://img.shields.io/github/release/shadowcz007/comfyui-mixlab-nodes)
 
-> 适配了最新版 comfyui 的 py3.11 ，torch 2.1.2+cu121
+> 适配了最新版 comfyui 的 py3.11 ，torch 2.3.1+cu121
 > [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)
 
 
 ##### `最新`：
 
-ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。模型下载后，放置到 `models/llamafile/`
+- App模式增加batch prompt，批量提示词，可以把动态提示词批量组成后运行
 
-- 右键菜单支持 text-to-text，方便对 prompt 词补全
+![alt text](./assets/1722517810720.png)
 
+- 增加 API Key Input 节点，用于管理LLM的Key,同时优化LLM相关节点，为后续agent模式做准备
+
+- 增加 SiliconflowLLM，可以使用由Siliconflow提供的免费LLM
+
+- 增加 Edit Mask，方便在生成的时候手动绘制 mask [workflow](./workflow/edit-mask-workflow.json)
+
+- LaMaInpainting 调整为手动安装
+
+<!-- - ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。模型下载后，放置到 `models/llamafile/` -->
+
+<!-- - 右键菜单支持 text-to-text，方便对 prompt 词补全 -->
+<!-- 
 强烈推荐：
 [Phi-3-mini-4k-instruct-function-calling-GGUF](https://huggingface.co/nold/Phi-3-mini-4k-instruct-function-calling-GGUF)
 
@@ -18,12 +30,16 @@ ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一
 - 右键菜单支持 image-to-text，使用多模态模型，多模态使用 [llava-phi-3-mini-gguf](https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main)，注意需要把llava-phi-3-mini-mmproj-f16.gguf也下载
 
 ![](./assets/prompt_ai_setup.png)
-![](./assets/prompt-ai.png)
+![](./assets/prompt-ai.png) -->
 
 
 #### `相关插件推荐`
 
-<!-- [comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab) -->
+[comfyui-liveportrait](https://github.com/shadowcz007/comfyui-liveportrait)
+
+[Comfyui-ChatTTS](https://github.com/shadowcz007/Comfyui-ChatTTS)
+
+[comfyui-sound-lab](https://github.com/shadowcz007/comfyui-sound-lab)
 
 [comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)
 
@@ -40,6 +56,8 @@ ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一
 - 发布为 app 的 workflow，可以在右键里再次编辑了
 - web app 可以设置分类，在 comfyui 右键菜单可以编辑更新 web app
 - 支持动态提示
+- 支持把输出显示到comfyui背景（TouchDesigner 风格）
+- 如果转为web app打开是空白的，注意检查下插件目录的名字需要是：comfyui-mixlab-nodes(如果是zip包下载会多了个-main的后缀，需要去掉)
 
 ![](./assets/微信图片_20240421205440.png)
 
@@ -98,15 +116,20 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 
 [Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
 
+- Preview Audio
+
+[text-to-audio](./workflow/text-to-audio-base-workflow.json)
+
 ### GPT
 
-> Support for calling multiple GPTs.Local LLM（llama.cpp）、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
+> Support for calling multiple GPTs.Local LLM 、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
 
-![gpt-workflow.svg](./assets/gpt-workflow.svg)
+[LLM_base_workflow](./workflow/LLM_base_workflow.json)
 
-[workflow-5](./workflow/5-gpt-workflow.json)
+- SiliconflowLLM
+- ChatGPTOpenAI
 
-最新：ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
+<!-- 最新：ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
 
 Model download,move to :`models/llamafile/`
 
@@ -134,7 +157,7 @@ pip install 'llama-cpp-python[server]'
 ```
 pip install llama-cpp-python \
   --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
-```
+``` -->
 
 ## Prompt
 
@@ -161,6 +184,9 @@ pip install llama-cpp-python \
 
 > A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.
 
+> The composite images node overlays a foreground image onto a background image at specified positions and scales, with optional blending modes and masking capabilities. position : 'overall',"center_center","left_bottom","center_bottom","right_bottom","left_top","center_top","right_top"
+
+
 ![layers](./assets/layers-workflow.svg)
 
 ![poster](./assets/poster-workflow.svg)
@@ -192,6 +218,12 @@ pip install llama-cpp-python \
 
 > Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.
 
+#### TextImage
+
+> [下载字体](https://drxie.github.io/OSFCC/)放到 ```custom_nodes/comfyui-mixlab-nodes/assets/fonts```
+
+
+
 ### Style
 
 > Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
@@ -229,10 +261,14 @@ Add edges to an image.
 
 ![FeatheredMask](./assets/FlVou_Y6kaGWYoEj1Tn0aTd4AjMI.jpg)
 
-> LaMaInpainting
+> LaMaInpainting（需要手动安装）
+
+*  simple-lama-inpainting 里的pillow造成冲突，暂时从依赖里移除，如果有安装 simple-lama-inpainting ，节点会自动添加，没有，则不会自动添加。
 
 from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainting)
 
+* [问题汇总](https://github.com/shadowcz007/comfyui-mixlab-nodes/issues/294)
+
 > rembgNode
 
 "briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"

__init__.py

@@ -3,12 +3,14 @@ import os
 import subprocess
 import importlib.util
 import sys,json
-import urllib
+import execution
+import uuid
 import hashlib
 import datetime
 import folder_paths
 import logging
 import base64,io,re
+import random
 from PIL import Image
 from comfy.cli_args import args
 python = sys.executable
@@ -20,16 +22,15 @@ except:
     print('#fix sys.stdout.isatty')
     sys.stdout.isatty = lambda: False
 
-llama_port=None
-llama_model=""
-llama_chat_format=""
+_URL_=None
 
-try:
-    from .nodes.ChatGPT import get_llama_models,get_llama_model_path,llama_cpp_client
-    llama_cpp_client("")
 
-except:
-    print("##nodes.ChatGPT ImportError")
+# try:
+#     from .nodes.ChatGPT import get_llama_models,get_llama_model_path,llama_cpp_client
+#     llama_cpp_client("")
+
+# except:
+#     print("##nodes.ChatGPT ImportError")
 
 
 from .nodes.RembgNode import get_rembg_models,U2NET_HOME,run_briarmbg,run_rembg
@@ -45,26 +46,35 @@ except ImportError:
     print("or")
     print("pip install -r requirements.txt")
     sys.exit()
-   
-def is_installed(package, package_overwrite=None):
+
+
+def is_installed(package, package_overwrite=None,auto_install=True):
+    is_has=False
     try:
         spec = importlib.util.find_spec(package)
+        is_has=spec is not None
     except ModuleNotFoundError:
         pass
 
     package = package_overwrite or package
 
     if spec is None:
-        print(f"Installing {package}...")
-        # 清华源 -i https://pypi.tuna.tsinghua.edu.cn/simple
-        command = f'"{python}" -m pip install {package}'
-  
-        result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, env=os.environ)
+        if auto_install==True:
+            print(f"Installing {package}...")
+            # 清华源 -i https://pypi.tuna.tsinghua.edu.cn/simple
+            command = f'"{python}" -m pip install {package}'
+    
+            result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, env=os.environ)
 
-        if result.returncode != 0:
-            print(f"Couldn't install\nCommand: {command}\nError code: {result.returncode}")
+            is_has=True
+
+            if result.returncode != 0:
+                print(f"Couldn't install\nCommand: {command}\nError code: {result.returncode}")
+                is_has=False
     else:
         print(package+'## OK')
+
+    return is_has
      
 try:
     import OpenSSL
@@ -87,7 +97,6 @@ except ImportError:
     sys.exit()
 
 
-
 def install_openai():
     # Helper function to install the OpenAI module if not already installed
     try:
@@ -171,15 +180,14 @@ def create_for_https():
         os.mkdir(https_key_path)
     if not os.path.exists(crt):
         create_key(key,crt)
-
-    print('https_key OK: ', crt,key)
+    # print('https_key OK: ', crt,key)
     return (crt,key)
 
 
 
 # workflow  目录下的所有json
 def read_workflow_json_files_all(folder_path):
-    print('#read_workflow_json_files_all',folder_path)
+    # print('#read_workflow_json_files_all',folder_path)
     json_files = []
     for root, dirs, files in os.walk(folder_path):
         for file in files:
@@ -309,31 +317,32 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
         print('app_workflow_path: ',app_workflow_path)
         try:
             with open(app_workflow_path) as json_file:
+                json_data=json.load(json_file)
                 apps = [{
                     'filename':filename,
-                    'data':json.load(json_file)
+                    'data':json_data
                 }]
         except Exception as e:
             print("发生异常：", str(e))
         
         # 这个代码不需要
         # if len(apps)==1 and category!='' and category!=None:
-            data=read_workflow_json_files(category_path)
+        data=read_workflow_json_files(category_path)
             
-            for item in data:
-                x=item["data"]
-                # print(apps[0]['filename'] ,item["filename"])
-                if apps[0]['filename']!=item["filename"]:
-                    category=''
-                    input=None
-                    output=None
-                    if 'category' in x['app']:
-                        category=x['app']['category']
-                    if 'input' in x['app']:
-                        input=x['app']['input']
-                    if 'output' in x['app']:
-                        output=x['app']['output']
-                    apps.append({
+        for item in data:
+            x=item["data"]
+            # print(apps[0]['filename'] ,item["filename"])
+            if apps[0]['filename']!=item["filename"]:
+                category=''
+                input=None
+                output=None
+                if 'category' in x['app']:
+                    category=x['app']['category']
+                if 'input' in x['app']:
+                    input=x['app']['input']
+                if 'output' in x['app']:
+                    output=x['app']['output']
+                apps.append({
                         "filename":item["filename"],
                         # "category":category,
                         "data":{
@@ -453,6 +462,7 @@ async def check_port_available(address, port):
 
 # https
 async def new_start(self, address, port, verbose=True, call_on_start=None):
+    global _URL_
     try:
         runner = web.AppRunner(self.app, access_log=None)
         await runner.setup()
@@ -521,10 +531,19 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
             logging.info("\n")
             logging.info("\n\nStarting server")
 
+            import socket
+
+            hostname = socket.gethostname()
+            ip_address = socket.gethostbyname(hostname)
+
+            # print(f"本机的IP地址是: {ip_address}")
+
+
             # print("\033[93mStarting server\n")
-            logging.info("\033[93mTo see the GUI go to: http://{}:{}".format(address, http_port))
-            logging.info("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))
- 
+            logging.info("\033[93mTo see the GUI go to: http://{}:{} or http://{}:{}".format(ip_address, http_port,address,http_port))
+            logging.info("\033[93mTo see the GUI go to: https://{}:{} or https://{}:{}\033[0m".format(ip_address, https_port,address,https_port))
+
+            _URL_="http://{}:{}".format(address,http_port)
             # print("\033[93mTo see the GUI go to: http://{}:{}".format(address, http_port))
             # print("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))
 
@@ -608,13 +627,34 @@ async def mixlab_workflow_hander(request):
                     category=data['category']
                 if 'admin' in data:
                     admin=data['admin']
+
+                ds=get_my_workflow_for_app(filename,category,admin)
+                data=[]           
+                for json_data in ds:
+                    # 不传给前端
+                    if 'output' in json_data['data']:
+                        del json_data['data']['output']
+                    if 'workflow' in json_data['data']:
+                        del json_data['data']['workflow']
+                    data.append(json_data)
+
                 result={
-                    'data':get_my_workflow_for_app(filename,category,admin),
+                    'data':data,
                     'status':'success',
                 }
             elif data['task']=='list':
+                ds=get_workflows()
+                data=[]
+                for json_data in ds:
+                    # 不传给前端
+                    if 'output' in json_data['data']:
+                        del json_data['data']['output']
+                    if 'workflow' in json_data['data']:
+                        del json_data['data']['workflow']
+                    data.append(json_data)
+
                 result={
-                    'data':get_workflows(),
+                    'data':data,
                     'status':'success',
                 }
     except Exception as e:
@@ -648,11 +688,11 @@ async def get_checkpoints(request):
     except Exception as e:
         print('/mixlab/folder_paths',False,e)
 
-    try:
-        if data['type']=='llamafile':
-            names=get_llama_models()
-    except:
-        print("llamafile none")
+    # try:
+    #     if data['type']=='llamafile':
+    #         names=get_llama_models()
+    # except:
+    #     print("llamafile none")
 
     try:
         if data['type']=='rembg':
@@ -699,135 +739,259 @@ async def rembg_hander(request):
 
     return web.json_response(result)
 
-
-@routes.post("/mixlab/prompt_result")
-async def post_prompt_result(request):
-    data = await request.json()
-    res=None
-    # print(data)
-    try:
-        action=data['action']
-        if action=='save':
-            result=data['data']
-            res=save_prompt_result(result['prompt_id'],result)
-        elif action=='all':
-            res=get_prompt_result()
-    except Exception as e:
-        print('/mixlab/prompt_result',False,e)
+# 保存运行结果？暂时去掉
+# @routes.post("/mixlab/prompt_result")
+# async def post_prompt_result(request):
+#     data = await request.json()
+#     res=None
+#     # print(data)
+#     try:
+#         action=data['action']
+#         if action=='save':
+#             result=data['data']
+#             res=save_prompt_result(result['prompt_id'],result)
+#         elif action=='all':
+#             res=get_prompt_result()
+#     except Exception as e:
+#         print('/mixlab/prompt_result',False,e)
     
-    return web.json_response({"result":res})
+#     return web.json_response({"result":res})
 
+# 种子设置
+def random_seed(seed, data):
+    max_seed = 4294967295
 
-async def start_local_llm(data):
-    global llama_port,llama_model,llama_chat_format
-    if llama_port and llama_model and llama_chat_format:
-        return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
-
-    import threading
-    import uvicorn
-    from llama_cpp.server.app import create_app
-    from llama_cpp.server.settings import (
-                Settings,
-                ServerSettings,
-                ModelSettings,
-                ConfigFileSettings,
-            )
+    for id, value in data.items():
+        # print(seed,id)
+        if id in seed:
+            if 'seed' in value['inputs'] and not isinstance(value['inputs']['seed'], list) and seed[id] in ['increment', 'decrement', 'randomize']:
+                value['inputs']['seed'] = round(random.random() * max_seed)
+            
+            if 'noise_seed' in value['inputs'] and not isinstance(value['inputs']['noise_seed'], list) and seed[id] in ['increment', 'decrement', 'randomize']:
+                value['inputs']['noise_seed'] = round(random.random() * max_seed)
+            
+            if value.get('class_type') == "Seed_" and seed[id] in ['increment', 'decrement', 'randomize']:
+                value['inputs']['seed'] = round(random.random() * max_seed)
+            
+        print('new Seed', value)
     
-    if not "model" in data and "model_path" in data:
-        data['model']= os.path.basename(data["model_path"])
-        model=data["model_path"]
-
-    elif "model" in data:
-        model=get_llama_model_path(data['model'])
-
-    n_gpu_layers=-1
-
-    if "n_gpu_layers" in data:
-        n_gpu_layers=data['n_gpu_layers']
+    return data
 
 
-    chat_format="chatml"
+# 运行工作流，代替官方的prompt接口
+@routes.post("/mixlab/prompt")
+async def mixlab_post_prompt(request):
+    p_intance=PromptServer.instance
+    logging.info("got prompt")
+    resp_code = 200
+    out_string = ""
+    json_data =  await request.json()
+    # json_data = p_intance.trigger_on_prompt(json_data)
+    # filename,category, client_id ,input
+    # workflow 的 filename,category
 
-    model_alias=os.path.basename(model)
+    # 输入的参数
+    input_data=json_data['input'] if "input" in json_data else []
+    # 种子
+    seed=json_data['seed'] if "seed" in json_data else {}
 
-    # 多模态
-    clip_model_path=None
+    apps=get_my_workflow_for_app(json_data['filename'],json_data['category'],False)
     
-    prefix = "llava-phi-3-mini"
-    file_name = prefix+"-mmproj-"
-    if model_alias.startswith(prefix):
-        for file in os.listdir(os.path.dirname(model)):
-            if file.startswith(file_name):
-                clip_model_path=os.path.join(os.path.dirname(model),file)
-                chat_format='llava-1-5'
-    print('#clip_model_path',chat_format,clip_model_path)
+    prompt=json_data['prompt'] if 'prompt' in json_data else None
+
+    if len(apps)==1:
+        # 取到prompt
+        prompt=apps[0]['data']['output']
+        # 更新input_data到prompt里
+        '''
+          {
+                "inputs": {
+                    "number": 512,
+                    "min_value": 512,
+                    "max_value": 2048,
+                    "step": 1
+                },
+                "class_type": "IntNumber",
+                "id": "22"
+            },
+        '''
+
+        for inp in input_data:
+            id=inp['id']
+            if prompt[id]['class_type']==inp['class_type']:
+                prompt[id]['inputs'].update(inp['inputs']) 
 
 
-    address="127.0.0.1"
-    port=9090
-    success = False
-    for i in range(11):  # 尝试最多11次
-        if await check_port_available(address, port + i):
-            port = port + i
-            success = True
-            break
+    if prompt==None:
+        return web.json_response({"error": "no prompt", "node_errors": []}, status=400)
+    else:
+        # 种子更新 
+        '''
+            "seed": {
+                    "45": "randomize",
+                    "46": "randomize"
+                }
+            '''
+        json_data["prompt"]=random_seed(seed,prompt)
 
-    if success == False:
-        return {"port":None,"model":""}
+    # print("#json_data",prompt)
+    # 需要把apps处理成 prompt
+    # 注意seed的处理
+    
+    if "number" in json_data:
+        number = float(json_data['number'])
+    else:
+        number = p_intance.number
+        if "front" in json_data:
+            if json_data['front']:
+                number = -number
+
+        p_intance.number += 1
+
+    if "prompt" in json_data:
+        prompt = json_data["prompt"]
+        valid = execution.validate_prompt(prompt)
+        extra_data = {}
+        if "extra_data" in json_data:
+            extra_data = json_data["extra_data"]
+
+        if "client_id" in json_data:
+            extra_data["client_id"] = json_data["client_id"]
+        if valid[0]:
+            prompt_id = str(uuid.uuid4())
+            outputs_to_execute = valid[2]
+            p_intance.prompt_queue.put((number, prompt_id, prompt, extra_data, outputs_to_execute))
+            response = {"prompt_id": prompt_id, "number": number, "node_errors": valid[3]}
+            return web.json_response(response)
+        else:
+            logging.warning("invalid prompt: {}".format(valid[1]))
+            return web.json_response({"error": valid[1], "node_errors": valid[3]}, status=400)
+    else:
+        return web.json_response({"error": "no prompt", "node_errors": []}, status=400)
+
+
+# AR页面
+# @routes.get('/mixlab/AR')
+async def handle_ar_page(request):
+    html_file = os.path.join(current_path, "web/ar.html")
+    if os.path.exists(html_file):
+        with open(html_file, 'r', encoding='utf-8', errors='ignore') as f:
+            html_data = f.read()
+            return web.Response(text=html_data, content_type='text/html')
+    else:
+        return web.Response(text="HTML file not found", status=404)
+
+
+# async def start_local_llm(data):
+#     global llama_port,llama_model,llama_chat_format
+#     if llama_port and llama_model and llama_chat_format:
+#         return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
+
+#     import threading
+#     import uvicorn
+#     from llama_cpp.server.app import create_app
+#     from llama_cpp.server.settings import (
+#                 Settings,
+#                 ServerSettings,
+#                 ModelSettings,
+#                 ConfigFileSettings,
+#             )
+    
+#     if not "model" in data and "model_path" in data:
+#         data['model']= os.path.basename(data["model_path"])
+#         model=data["model_path"]
+
+#     elif "model" in data:
+#         model=get_llama_model_path(data['model'])
+
+#     n_gpu_layers=-1
+
+#     if "n_gpu_layers" in data:
+#         n_gpu_layers=data['n_gpu_layers']
+
+
+#     chat_format="chatml"
+
+#     model_alias=os.path.basename(model)
+
+#     # 多模态
+#     clip_model_path=None
+    
+#     prefix = "llava-phi-3-mini"
+#     file_name = prefix+"-mmproj-"
+#     if model_alias.startswith(prefix):
+#         for file in os.listdir(os.path.dirname(model)):
+#             if file.startswith(file_name):
+#                 clip_model_path=os.path.join(os.path.dirname(model),file)
+#                 chat_format='llava-1-5'
+#     # print('#clip_model_path',chat_format,clip_model_path,model)
+
+#     address="127.0.0.1"
+#     port=9090
+#     success = False
+#     for i in range(11):  # 尝试最多11次
+#         if await check_port_available(address, port + i):
+#             port = port + i
+#             success = True
+#             break
+
+#     if success == False:
+#         return {"port":None,"model":""}
     
     
-    server_settings=ServerSettings(host=address,port=port)
+#     server_settings=ServerSettings(host=address,port=port)
 
-    name, ext = os.path.splitext(os.path.basename(model))
-    print('#model',name)
-    app = create_app(
-                server_settings=server_settings,
-                model_settings=[
-                    ModelSettings(
-                    model=model,
-                    model_alias=name,
-                    n_gpu_layers=n_gpu_layers,
-                    n_ctx=4098,
-                    chat_format=chat_format,
-                    embedding=False,
-                    clip_model_path=clip_model_path
-                    )])
+#     name, ext = os.path.splitext(os.path.basename(model))
+#     if name:
+#         # print('#model',name)
+#         app = create_app(
+#                     server_settings=server_settings,
+#                     model_settings=[
+#                         ModelSettings(
+#                         model=model,
+#                         model_alias=name,
+#                         n_gpu_layers=n_gpu_layers,
+#                         n_ctx=4098,
+#                         chat_format=chat_format,
+#                         embedding=False,
+#                         clip_model_path=clip_model_path
+#                         )])
 
-    def run_uvicorn():
-        uvicorn.run(
-                app,
-                host=os.getenv("HOST", server_settings.host),
-                port=int(os.getenv("PORT", server_settings.port)),
-                ssl_keyfile=server_settings.ssl_keyfile,
-                ssl_certfile=server_settings.ssl_certfile,
-            )
+#         def run_uvicorn():
+#             uvicorn.run(
+#                     app,
+#                     host=os.getenv("HOST", server_settings.host),
+#                     port=int(os.getenv("PORT", server_settings.port)),
+#                     ssl_keyfile=server_settings.ssl_keyfile,
+#                     ssl_certfile=server_settings.ssl_certfile,
+#                 )
 
-    # 创建一个子线程
-    thread = threading.Thread(target=run_uvicorn)
+#         # 创建一个子线程
+#         thread = threading.Thread(target=run_uvicorn)
 
-    # 启动子线程
-    thread.start()
+#         # 启动子线程
+#         thread.start()
 
-    llama_port=port
-    llama_model=data['model']
-    llama_chat_format=chat_format
+#         llama_port=port
+#         llama_model=data['model']
+#         llama_chat_format=chat_format
 
-    return  {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
+#     return  {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
 
 # llam服务的开启
-@routes.post('/mixlab/start_llama')
-async def my_hander_method(request):
-    data =await request.json()
-    # print(data)
-    if llama_port and llama_model and llama_chat_format:
-        return web.json_response({"port":llama_port,"model":llama_model,"chat_format":llama_chat_format} )
-    try:
-        result=await start_local_llm(data)
-    except:
-        result= {"port":None,"model":"","llama_cpp_error":True}
-        print('start_local_llm error')
+# @routes.post('/mixlab/start_llama')
+# async def my_hander_method(request):
+#     data =await request.json()
+#     # print(data)
+#     if llama_port and llama_model and llama_chat_format:
+#         return web.json_response({"port":llama_port,"model":llama_model,"chat_format":llama_chat_format} )
+#     try:
+#         result=await start_local_llm(data)
+#     except:
+#         result= {"port":None,"model":"","llama_cpp_error":True}
+#         print('start_local_llm error')
 
-    return web.json_response(result)
+#     return web.json_response(result)
 
 # 重启服务
 @routes.post('/mixlab/re_start')
@@ -838,24 +1002,24 @@ def re_start(request):
         pass
     return os.execv(sys.executable, [sys.executable] + sys.argv)
 
-
+# 状态
+@routes.get('/mixlab/status')
+def mix_status(request):
+    return web.Response(text="running#"+_URL_)
 
 # 导入节点
 from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
-from .nodes.ImageNode import ComparingTwoFrames,LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
+from .nodes.ImageNode import ImageBatchToList_,ImageListToBatch_,ComparingTwoFrames,LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
 # from .nodes.Vae import VAELoader,VAEDecode
 from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
 
-from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText,TextSplitByDelimiter
-from .nodes.Audio import GamePal,SpeechRecognition,SpeechSynthesis
-from .nodes.Utils import IncrementingListNode,ListSplit,CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,TESTNODE_TOKEN,AppInfo,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
+from .nodes.Audio import AudioPlayNode,SpeechRecognition,SpeechSynthesis
+from .nodes.Utils import KeyInput,IncrementingListNode,ListSplit,CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,TESTNODE_TOKEN,AppInfo,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
 from .nodes.Mask import PreviewMask_,MaskListReplace,MaskListMerge,OutlineMask,FeatheredMask
 
 from .nodes.Style import ApplyVisualStylePrompting,StyleAlignedReferenceSampler,StyleAlignedBatchAlign,StyleAlignedSampleReferenceLatents
 
-from .nodes.Video import VideoCombine_Adv,LoadVideoAndSegment,ImageListReplace,VAEEncodeForInpaint_Frames
-
-from .nodes.TripoSR import LoadTripoSRModel,TripoSRSampler,SaveTripoSRMesh
+from .nodes.P5 import P5Input
 
 
 # 要导出的所有节点及其名称的字典
@@ -886,6 +1050,8 @@ NODE_CLASS_MAPPINGS = {
     "ImageColorTransfer":ImageColorTransfer,
     "ShowLayer":ShowLayer,
     "NewLayer":NewLayer,
+    "ImageListToBatch_":ImageListToBatch_,
+    "ImageBatchToList_":ImageBatchToList_,
     "CompositeImages_":CompositeImages,
     "SplitImage":SplitImage,
     "CenterImage":CenterImage,
@@ -907,12 +1073,10 @@ NODE_CLASS_MAPPINGS = {
     # "VAEDecodeConsistencyDecoder":VAEDecode,
     "ScreenShare":ScreenShareNode,
     "FloatingVideo":FloatingVideo,
-    "ChatGPTOpenAI":ChatGPTNode,
-    "ShowTextForGPT":ShowTextForGPT,
-    "CharacterInText":CharacterInText,
-    "TextSplitByDelimiter":TextSplitByDelimiter,
+   
     "SpeechRecognition":SpeechRecognition,
     "SpeechSynthesis":SpeechSynthesis,
+    "KeyInput":KeyInput,
     "Color":ColorInput,
     "FloatSlider":FloatSlider,
     "IntNumber":IntNumber,
@@ -934,26 +1098,26 @@ NODE_CLASS_MAPPINGS = {
     "ApplyVisualStylePrompting_":ApplyVisualStylePrompting,
     "StyleAlignedReferenceSampler_": StyleAlignedReferenceSampler,
     "StyleAlignedSampleReferenceLatents_": StyleAlignedSampleReferenceLatents,
-    "StyleAlignedBatchAlign_": StyleAlignedBatchAlign,
-    "LoadVideoAndSegment_":LoadVideoAndSegment,
-    "VideoCombine_Adv":VideoCombine_Adv,
+    "StyleAlignedBatchAlign_": StyleAlignedBatchAlign, 
     "ListSplit_":ListSplit,
-    "MaskListReplace_":MaskListReplace,
-    "ImageListReplace_":ImageListReplace,
-    "VAEEncodeForInpaint_Frames":VAEEncodeForInpaint_Frames,
+    "MaskListReplace_":MaskListReplace, 
     "IncrementingListNode_":IncrementingListNode,
     "PreviewMask_":PreviewMask_,
-     "LoadTripoSRModel_": LoadTripoSRModel,
-    "TripoSRSampler_": TripoSRSampler,
-    "SaveTripoSRMesh": SaveTripoSRMesh
-    # "GamePal":GamePal
+    "AudioPlay":AudioPlayNode,
+
+    "P5Input":P5Input
 }
 
 # 一个包含节点友好/可读的标题的字典
 NODE_DISPLAY_NAME_MAPPINGS = {
     "AppInfo":"App Info ♾️MixlabApp",
+    "ScreenShare":"Screen Share ♾️Mixlab",
+    "FloatingVideo":"Floating Video ♾️Mixlab",
+    "TextImage":"Text Image ♾️Mixlab",
+    
     "Color":"Color Input ♾️MixlabApp",
     "TextInput_":"Text Input ♾️MixlabApp",
+    "KeyInput":"API Key Input ♾️MixlabApp",
     "FloatSlider":"Float Slider Input ♾️MixlabApp",
     "IntNumber":"Int Input ♾️MixlabApp",
     "ImagesPrompt_":"Images Input ♾️MixlabApp",
@@ -965,14 +1129,14 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "SplitLongMask":"Splitting a long image into sections",
     "VAELoaderConsistencyDecoder":"Consistency Decoder Loader",
     "VAEDecodeConsistencyDecoder":"Consistency Decoder Decode",
-    "ScreenShare":"Screen Share ♾️Mixlab",
-    "FloatingVideo":"FloatingVideo ♾️Mixlab",
-    "ChatGPTOpenAI":"ChatGPT & Local LLM ♾️Mixlab",
-    "ShowTextForGPT":"Show Text ♾️MixlabApp",
+    
+    
     "MergeLayers":"Merge Layers ♾️Mixlab",
     "SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
     "SpeechRecognition":"SpeechRecognition ♾️Mixlab",
     "3DImage":"3DImage ♾️Mixlab",
+    "ImageListToBatch_":"Image List To Batch",
+    "ImageBatchToList_":"Image Batch To List",
     "CompositeImages_":"Composite Images ♾️Mixlab",
     "DynamicDelayProcessor":"DynamicDelayByText ♾️Mixlab",
     "LaMaInpainting":"LaMaInpainting ♾️Mixlab",
@@ -998,28 +1162,69 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "GridInput":"Grid Input ♾️Mixlab",
     "GridOutput":"Grid Output ♾️Mixlab",
     "GetImageSize_":"Get Image Size ♾️Mixlab",
-    "VAEEncodeForInpaint_Frames":"VAE Encode For Inpaint Frames ♾️Mixlab",
     "IncrementingListNode_":"Create Incrementing Number List ♾️Mixlab",
     "LoadImagesToBatch":"Load Images(base64) ♾️Mixlab",
     "PreviewMask_":"Preview Mask",
-    "LoadTripoSRModel_": "Load TripoSR Model",
-    "TripoSRSampler_": "TripoSR Sampler",
-    "SaveTripoSRMesh": "Save TripoSR Mesh"
+    "AudioPlay":"Preview Audio ♾️Mixlab",
+
+     "MultiplicationNode":"Math Operation ♾️Mixlab",
+
+     "P5Input":"P5 Input ♾️Mixlab for test"
 }
 
 # web ui的节点功能
 WEB_DIRECTORY = "./web"
 
-
 logging.info('--------------')
 logging.info('\033[91m ### Mixlab Nodes: \033[93mLoaded')
 # print('\033[91m ### Mixlab Nodes: \033[93mLoaded')
 
 try:
-    from .nodes.Lama import LaMaInpainting
-    logging.info('LaMaInpainting.available {}'.format(LaMaInpainting.available))
-    if LaMaInpainting.available:
-        NODE_CLASS_MAPPINGS['LaMaInpainting']=LaMaInpainting
+    from .nodes.ChatGPT import JsonRepair,ChatGPTNode,ShowTextForGPT,CharacterInText,TextSplitByDelimiter,SiliconflowFreeNode
+    logging.info('ChatGPT.available True')
+
+    NODE_CLASS_MAPPINGS_V = { 
+       "ChatGPTOpenAI":ChatGPTNode,
+       "SiliconflowLLM":SiliconflowFreeNode,
+        "ShowTextForGPT":ShowTextForGPT,
+        "CharacterInText":CharacterInText,
+        "TextSplitByDelimiter":TextSplitByDelimiter,
+        "JsonRepair":JsonRepair
+    }
+
+    # 一个包含节点友好/可读的标题的字典
+    NODE_DISPLAY_NAME_MAPPINGS_V = { 
+        "ChatGPTOpenAI":"ChatGPT & Local LLM ♾️Mixlab",
+        "SiliconflowLLM":"LLM Siliconflow ♾️Mixlab",
+        "ShowTextForGPT":"Show Text ♾️MixlabApp",
+        "CharacterInText":"Character In Text",
+        "TextSplitByDelimiter":"Text Split By Delimiter",
+        "JsonRepair":"Json Repair"
+    }
+
+
+    NODE_CLASS_MAPPINGS.update(NODE_CLASS_MAPPINGS_V)
+    NODE_DISPLAY_NAME_MAPPINGS.update(NODE_DISPLAY_NAME_MAPPINGS_V)
+
+except Exception as e:
+    logging.info('ChatGPT.available False')
+
+
+try:
+    from .nodes.edit_mask import EditMask
+    logging.info('edit_mask.available True')
+    NODE_CLASS_MAPPINGS['EditMask']=EditMask
+    NODE_DISPLAY_NAME_MAPPINGS['EditMask']="Edit Mask ♾️Mixlab"
+except Exception as e:
+    logging.info('edit_mask.available False')
+
+try:
+    is_has=is_installed('simple_lama_inpainting',None,False)
+    if is_has:
+        from .nodes.Lama import LaMaInpainting
+        logging.info('LaMaInpainting.available {}'.format(LaMaInpainting.available))
+        if LaMaInpainting.available:
+            NODE_CLASS_MAPPINGS['LaMaInpainting']=LaMaInpainting
 except Exception as e:
     logging.info('LaMaInpainting.available False')
 
@@ -1050,4 +1255,57 @@ try:
 except Exception as e:
     logging.info('RembgNode_.available False' )
 
+
+try:
+    from .nodes.Video import GenerateFramesByCount,scenesNode_,CombineAudioVideo,VideoCombine_Adv,LoadVideoAndSegment,ImageListReplace,VAEEncodeForInpaint_Frames,LoadAndCombinedAudio_
+    
+    NODE_CLASS_MAPPINGS_V = {
+        "VAEEncodeForInpaint_Frames":VAEEncodeForInpaint_Frames,
+        "ImageListReplace_":ImageListReplace,
+        "LoadVideoAndSegment_":LoadVideoAndSegment,
+        "VideoCombine_Adv":VideoCombine_Adv,
+        "LoadAndCombinedAudio_":LoadAndCombinedAudio_,
+        "CombineAudioVideo":CombineAudioVideo,
+        "ScenesNode_":scenesNode_,
+        "GenerateFramesByCount":GenerateFramesByCount 
+    }
+
+    # 一个包含节点友好/可读的标题的字典
+    NODE_DISPLAY_NAME_MAPPINGS_V = {
+        "VAEEncodeForInpaint_Frames":"VAE Encode For Inpaint Frames ♾️Mixlab",
+        "ImageListReplace_":"Image List Replace",
+        "LoadVideoAndSegment_":"Load Video And Segment",
+        "VideoCombine_Adv":"Video Combine",
+        "LoadAndCombinedAudio_":"Load And Combined Audio",
+        "CombineAudioVideo":"Combine Audio Video",
+        "ScenesNode_":"Select Scene", 
+        "GenerateFramesByCount":"Generate Frames By Count"
+    }
+
+
+    NODE_CLASS_MAPPINGS.update(NODE_CLASS_MAPPINGS_V)
+    NODE_DISPLAY_NAME_MAPPINGS.update(NODE_DISPLAY_NAME_MAPPINGS_V)
+
+except:
+    logging.info('Video.available False')
+
+
+try:
+    from .nodes.TripoSR import LoadTripoSRModel,TripoSRSampler,SaveTripoSRMesh
+    logging.info('TripoSR.available')
+    # logging.info( folder_paths.get_temp_directory())
+    NODE_CLASS_MAPPINGS['LoadTripoSRModel_']=LoadTripoSRModel
+    NODE_DISPLAY_NAME_MAPPINGS["LoadTripoSRModel_"]= "Load TripoSR Model"
+    
+    NODE_CLASS_MAPPINGS['TripoSRSampler_']=TripoSRSampler
+    NODE_DISPLAY_NAME_MAPPINGS["TripoSRSampler_"]= "TripoSR Sampler"
+
+    NODE_CLASS_MAPPINGS['SaveTripoSRMesh']=SaveTripoSRMesh
+    NODE_DISPLAY_NAME_MAPPINGS["SaveTripoSRMesh"]= "Save TripoSR Mesh"
+    
+
+except Exception as e:
+    logging.info('TripoSR.available False' )
+
+
 logging.info('\033[93m -------------- \033[0m')

install.bat

@@ -11,9 +11,9 @@ if exist "%python_exec%" (
         %python_exec% -s -m pip install "%%i" -i https://pypi.tuna.tsinghua.edu.cn/simple
     )
 
-    %python_exec% -s -m pip install --upgrade --force llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
+    @REM %python_exec% -s -m pip install --upgrade --force llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
 
-    %python_exec% -s -m pip install --upgrade --force llama-cpp-python[server]
+    @REM %python_exec% -s -m pip install --upgrade --force llama-cpp-python[server]
     
 
 ) else (

nodes/Audio.py

@@ -1,6 +1,7 @@
 
-
-
+import os
+import folder_paths
+import torchaudio
 
 class SpeechRecognition:
     @classmethod
@@ -55,46 +56,65 @@ class SpeechSynthesis:
         return {"ui": {"text": text}, "result": (text,)}
     
 
-#
-class GamePal:
+
+class AudioPlayNode:
+    def __init__(self):
+        self.output_dir = folder_paths.get_temp_directory()
+        self.type = "temp"
+        self.prefix_append = ""
+        self.compress_level = 4
     @classmethod
     def INPUT_TYPES(s):
-        return {
-            "required": {
-                "input_text": ("STRING",{"multiline": True,"default": ""}),
-            },
-            "optional": {
-                 
-                "input_num": ("INT",{
-                                "default":100, 
-                                "min": -1, #Minimum value
-                                "max": 0xffffffffffffffff, #Maximum value
-                                "step": 1, #Slider's step
-                                "display": "slider" # Cosmetic only: display as "number" or "slider"
-                                }), 
-                "python_code": ("STRING",{"multiline": True,"default": "result= 1 if 'Mixlab' in input_text else 0"}),
-            }
-        }
-
-    INPUT_IS_LIST = False
-    RETURN_TYPES = ("INT",)
+        return {"required": {
+            "audio": ("AUDIO",),
+              }, 
+                }
+    
+    RETURN_TYPES = ()
+  
     FUNCTION = "run"
-    OUTPUT_NODE = True
-    OUTPUT_IS_LIST = (False,)
 
     CATEGORY = "♾️Mixlab/Audio"
 
-    def run(self, input_text,input_num,python_code):
-        exec(python_code)
-        res=None
-        try:
-            # 可能会引发异常的代码
-            res=result
-        except:
-            # 处理异常的代码
-            print('')
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = ()
 
-        print(res)
+    OUTPUT_NODE = True
+  
+    def run(self,audio):
 
-        # print(session_history)
-        return {"ui": {"text": [input_text],"num":[input_num]}, "result": (res,)}
+        # 判断是否是 Tensor 类型
+        is_tensor = not isinstance(audio, dict)
+        # print('#判断是否是 Tensor 类型',is_tensor,audio)
+        if not is_tensor and 'waveform' in audio and 'sample_rate' in audio:
+            # {'waveform': tensor([], size=(1, 1, 0)), 'sample_rate': 44100}
+            is_tensor=True
+
+        if is_tensor and (not 'audio_path' in audio):
+            filename_prefix=""
+            # 保存
+            filename_prefix += self.prefix_append
+            full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir)
+            results = list()
+            
+            filename_with_batch_num = filename.replace("%batch_num%", str(1))
+            file = f"{filename_with_batch_num}_{counter:05}_.wav"
+            
+            torchaudio.save(os.path.join(full_output_folder, file), audio['waveform'].squeeze(0), audio["sample_rate"])
+            results.append({
+                    "filename": file,
+                    "subfolder": subfolder,
+                    "type": self.type
+                })
+            
+        else:
+            results=[{
+                    "filename": audio['filename'],
+                    "subfolder":audio['subfolder'],
+                    "type": audio['type'],
+                    "audio_path":audio['audio_path']
+                }]
+                
+
+        # print(audio)
+        return {"ui": {"audio":results}}

nodes/ChatGPT.py

@@ -6,14 +6,69 @@ import folder_paths
 import hashlib
 import codecs,sys
 import importlib.util
+import subprocess
+
+python = sys.executable
+
+# 从文本中提取json
+def extract_json_strings(text):
+    json_strings = []
+    brace_level = 0
+    json_str = ''
+    in_json = False
+    
+    for char in text:
+        if char == '{':
+            brace_level += 1
+            in_json = True
+        if in_json:
+            json_str += char
+        if char == '}':
+            brace_level -= 1
+        if in_json and brace_level == 0:
+            json_strings.append(json_str)
+            json_str = ''
+            in_json = False
+
+    return json_strings[0] if len(json_strings)>0 else "{}"
 
 
-def is_installed(package):
+def is_installed(package, package_overwrite=None,auto_install=True):
+    is_has=False
     try:
         spec = importlib.util.find_spec(package)
+        is_has=spec is not None
     except ModuleNotFoundError:
-        return False
-    return spec is not None
+        pass
+
+    package = package_overwrite or package
+
+    if spec is None:
+        if auto_install==True:
+            print(f"Installing {package}...")
+            # 清华源 -i https://pypi.tuna.tsinghua.edu.cn/simple
+            command = f'"{python}" -m pip install {package}'
+    
+            result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, env=os.environ)
+
+            is_has=True
+
+            if result.returncode != 0:
+                print(f"Couldn't install\nCommand: {command}\nError code: {result.returncode}")
+                is_has=False
+    else:
+        print(package+'## OK')
+
+    return is_has
+  
+
+
+# def is_installed(package):
+#     try:
+#         spec = importlib.util.find_spec(package)
+#     except ModuleNotFoundError:
+#         return False
+#     return spec is not None
 
 
 def get_unique_hash(string):
@@ -53,30 +108,14 @@ def azure_client(key,url):
 
 def openai_client(key,url):
     client = openai.OpenAI(
-    api_key=key,
-    base_url=url
+        api_key=key,
+        base_url=url
     )
     return client
 
 def ZhipuAI_client(key):
-
     try:
-        if is_installed('zhipuai')==False:
-            import subprocess
-
-            # 安装
-            print('#pip install zhipuai')
-
-            result = subprocess.run([sys.executable, '-s', '-m', 'pip', 'install', 'zhipuai'], capture_output=True, text=True)
-
-            #检查命令执行结果
-            if result.returncode == 0:
-                print("#install success")
-                from zhipuai import ZhipuAI
-            else:
-                print("#install error")
-            
-        else:
+        if is_installed('zhipuai')==True:
             from zhipuai import ZhipuAI
     except:
         print("#install zhipuai error")
@@ -97,73 +136,76 @@ def get_llama_path():
     except:
         return os.path.join(folder_paths.models_dir, "llamafile")
 
-def get_llama_models():
-    res=[]
+# def get_llama_models():
+#     res=[]
 
-    model_path=get_llama_path()
-    if os.path.exists(model_path):
-        files = os.listdir(model_path)
-        for file in files:
-            if os.path.isfile(os.path.join(model_path, file)):
-                res.append(file)
-        res=phi_sort(res)
-    return res
+#     model_path=get_llama_path()
+#     if os.path.exists(model_path):
+#         files = os.listdir(model_path)
+#         for file in files:
+#             if os.path.isfile(os.path.join(model_path, file)):
+#                 res.append(file)
+#         res=phi_sort(res)
+#     return res
 
-llama_modes_list=get_llama_models()
+# llama_modes_list=get_llama_models()
+# llama_modes_list=[]
 
-def get_llama_model_path(file_name):
-    model_path=get_llama_path()
-    mp=os.path.join(model_path,file_name)
-    return mp
+# def get_llama_model_path(file_name):
+#     model_path=get_llama_path()
+#     mp=os.path.join(model_path,file_name)
+#     return mp
 
-def llama_cpp_client(file_name):
-    try:
-        if is_installed('llama_cpp')==False:
-            import subprocess
+# def llama_cpp_client(file_name):
+#     try:
+#         if is_installed('llama_cpp')==False:
+#             import subprocess
 
-            # 安装
-            print('#pip install llama-cpp-python')
+#             # 安装
+#             print('#pip install llama-cpp-python')
  
-            result = subprocess.run([sys.executable, '-s', '-m', 'pip', 
-                                     'install', 
-                                     'llama-cpp-python',
-                                     '--extra-index-url',
-                                     'https://abetlen.github.io/llama-cpp-python/whl/cu121'
-                                     ], capture_output=True, text=True)
+#             result = subprocess.run([sys.executable, '-s', '-m', 'pip', 
+#                                      'install', 
+#                                      'llama-cpp-python',
+#                                      '--extra-index-url',
+#                                      'https://abetlen.github.io/llama-cpp-python/whl/cu121'
+#                                      ], capture_output=True, text=True)
 
-            #检查命令执行结果
-            if result.returncode == 0:
-                print("#install success")
-                from llama_cpp import Llama
+#             #检查命令执行结果
+#             if result.returncode == 0:
+#                 print("#install success")
+#                 from llama_cpp import Llama
 
-                subprocess.run([sys.executable, '-s', '-m', 'pip', 
-                                     'install', 
-                                     'llama-cpp-python[server]'
-                                     ], capture_output=True, text=True)
+#                 subprocess.run([sys.executable, '-s', '-m', 'pip', 
+#                                      'install', 
+#                                      'llama-cpp-python[server]'
+#                                      ], capture_output=True, text=True)
 
-            else:
-                print("#install error")
+#             else:
+#                 print("#install error")
             
-        else:
-            from llama_cpp import Llama
-    except:
-        print("#install llama-cpp-python error")
+#         else:
+#             from llama_cpp import Llama
+#     except:
+#         print("#install llama-cpp-python error")
 
-    if file_name:
-        mp=get_llama_model_path(file_name)
-        # file_name=get_llama_models()[0]
-        # model_path=os.path.join(folder_paths.models_dir, "llamafile")
-        # mp=os.path.join(model_path,file_name)
+#     if file_name:
+#         mp=get_llama_model_path(file_name)
+#         # file_name=get_llama_models()[0]
+#         # model_path=os.path.join(folder_paths.models_dir, "llamafile")
+#         # mp=os.path.join(model_path,file_name)
 
-        llm = Llama(model_path=mp, chat_format="chatml",n_gpu_layers=-1,n_ctx=512)
+#         llm = Llama(model_path=mp, chat_format="chatml",n_gpu_layers=-1,n_ctx=512)
 
-        return llm
+#         return llm
 
-    
+
+if is_installed('json_repair'):
+    from json_repair import repair_json
 
 
 def chat(client, model_name,messages ):
-
+        print('#chat',model_name,messages)
         try_count = 0
         while True:
             try_count += 1
@@ -206,6 +248,36 @@ def chat(client, model_name,messages ):
         return content
 
 
+llm_apis=[
+            {
+                "value": "https://api.openai.com/v1",
+                "label": "openai"
+            },
+            {
+                "value": "https://openai.api2d.net/v1",
+                "label": "api2d"
+            },
+            # {
+            #     "value": "https://docs-test-001.openai.azure.com",
+            #     "label": "https://docs-test-001.openai.azure.com"
+            # },
+             
+            {
+                "value": "https://api.moonshot.cn/v1",
+                "label": "Kimi"
+            },
+            {
+                "value": "https://api.deepseek.com/v1",
+                "label": "DeepSeek-V2"
+            },
+            {
+                "value": "https://api.siliconflow.cn/v1",
+                "label": "SiliconCloud"
+            }]
+        
+llm_apis_dict = {api["label"]: api["value"] for api in llm_apis}
+
+
 class ChatGPTNode:
     def __init__(self):
         # self.__client = OpenAI()
@@ -215,35 +287,60 @@ class ChatGPTNode:
 
     @classmethod
     def INPUT_TYPES(cls):
-        model_list=llama_modes_list+[
-                    "gpt-3.5-turbo",
-                    "gpt-3.5-turbo-0125",
-                    "gpt-35-turbo",
-                    "gpt-3.5-turbo-16k", 
-                    "gpt-3.5-turbo-16k-0613", 
-                    "gpt-4-0613",
-                    "gpt-4-1106-preview",
-                    "glm-4"
+
+        model_list=[ 
+            "gpt-3.5-turbo",
+            "gpt-3.5-turbo-16k",
+            "gpt-4o",
+            "gpt-4o-2024-05-13",
+            "gpt-4",
+            "gpt-4-0314",
+            "gpt-4-0613",
+            "gpt-3.5-turbo-0301",
+            "gpt-3.5-turbo-0613",
+            "gpt-3.5-turbo-16k-0613",
+            "qwen-turbo",
+            "qwen-plus",
+            "qwen-long",
+            "qwen-max",
+            "qwen-max-longcontext",
+            "glm-4",
+            "glm-3-turbo",
+            "moonshot-v1-8k",
+            "moonshot-v1-32k",
+            "moonshot-v1-128k",
+            "deepseek-chat",
+            "Qwen/Qwen2-7B-Instruct",
+            "THUDM/glm-4-9b-chat",
+            "01-ai/Yi-1.5-9B-Chat-16K",
+            "meta-llama/Meta-Llama-3.1-8B-Instruct"
                     ]
+        
         return {
             "required": {
-                "api_key":("KEY", {"default": "", "multiline": True,"dynamicPrompts": False}),
-                "api_url":("URL", {"default": "", "multiline": True,"dynamicPrompts": False}),
+                # "api_key":("KEY", {"default": "", "multiline": True,"dynamicPrompts": False}),
+                # "api_key":("STRING", {"forceInput": True,}),
+               
                 "prompt": ("STRING", {"multiline": True,"dynamicPrompts": False}),
                 "system_content": ("STRING", 
                                    {
                                        "default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.", 
                                        "multiline": True,"dynamicPrompts": False
                                        }),
+                 
                 "model": ( model_list, 
                     {"default": model_list[0]}),
                 "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "step": 1}),
                 "context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
+                "api_url":(list(llm_apis_dict.keys()), 
+                    {"default": list(llm_apis_dict.keys())[0]}),
             },
-             "hidden": {
-                "unique_id": "UNIQUE_ID",
-                "extra_pnginfo": "EXTRA_PNGINFO",
-            },
+             "optional":{
+                    "api_key":("STRING", {"forceInput": True,}),
+                    "custom_model_name":("STRING", {"forceInput": True,}), #适合自定义model
+                     "custom_api_url":("STRING", {"forceInput": True,}), #适合自定义model
+                },
+             
         }
 
     RETURN_TYPES = ("STRING","STRING","STRING",)
@@ -255,12 +352,29 @@ class ChatGPTNode:
 
     
     def generate_contextual_text(self,
-                                 api_key,
-                                 api_url, 
+                                #  api_key,
                                  prompt, 
                                  system_content,
-                                model, 
-                                seed,context_size,unique_id = None, extra_pnginfo=None):
+                                 model, 
+                                seed,
+                                context_size,
+                                api_url,
+                                api_key=None,
+                                custom_model_name=None,
+                                custom_api_url=None,
+                                ):
+        
+        if custom_model_name!=None:
+            model=custom_model_name
+
+        api_url=llm_apis_dict[api_url] if api_url in llm_apis_dict else ""
+
+        if custom_api_url!=None:
+            api_url=custom_api_url
+
+        if api_key==None:
+            api_key="lm_studio"
+
         # print(api_key!='',api_url,prompt,system_content,model,seed)
         # 可以选择保留会话历史以维持上下文记忆
         # 或者在此处清除会话历史 self.session_history.clear()
@@ -273,7 +387,7 @@ class ChatGPTNode:
             self.system_content=system_content
             # self.session_history=[]
             # self.session_history.append({"role": "system", "content": system_content})
-        
+        print("api_key,api_url",api_key,api_url)
         # 
         if is_azure_url(api_url):
             client=azure_client(api_key,api_url)
@@ -282,12 +396,12 @@ class ChatGPTNode:
             if model == "glm-4" :
                 client = ZhipuAI_client(api_key)  # 使用 Zhipuai 的接口
                 print('using Zhipuai interface')
-            elif model in llama_modes_list:
-                #
-                client=llama_cpp_client(model)
+            # elif model in llama_modes_list:
+            #     #
+            #     client=llama_cpp_client(model)
             else :
                 client = openai_client(api_key,api_url)  # 使用 ChatGPT  的接口
-                print('using ChatGPT interface')
+                # print('using ChatGPT interface',api_key,api_url)
 
         # 把用户的提示添加到会话历史中
         # 调用API时传递整个会话历史
@@ -303,6 +417,7 @@ class ChatGPTNode:
         session_history=crop_list_tail(self.session_history,context_size)
 
         messages=[{"role": "system", "content": self.system_content}]+session_history+[{"role": "user", "content": prompt}]
+
         response_content = chat(client,model,messages)
         
         self.session_history=self.session_history+[{"role": "user", "content": prompt}]+[{'role':'assistant',"content":response_content}]
@@ -323,6 +438,93 @@ class ChatGPTNode:
         return (response_content,json.dumps(messages, indent=4),json.dumps(self.session_history, indent=4),)
 
 
+class SiliconflowFreeNode:
+    def __init__(self):
+        # self.__client = OpenAI()
+        self.session_history = []  # 用于存储会话历史的列表
+        # self.seed=0
+        self.system_content="You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible."
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        model_list= [ 
+            "Qwen/Qwen2-7B-Instruct",
+            "THUDM/glm-4-9b-chat",
+            "01-ai/Yi-1.5-9B-Chat-16K",
+            "meta-llama/Meta-Llama-3.1-8B-Instruct"
+            ]
+        return {
+            "required": {
+                "api_key":("STRING", {"forceInput": True,}),
+                "prompt": ("STRING", {"multiline": True,"dynamicPrompts": False}),
+                "system_content": ("STRING", 
+                                   {
+                                       "default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.", 
+                                       "multiline": True,"dynamicPrompts": False
+                                       }),
+                "model": ( model_list, 
+                    {"default": model_list[0]}),
+                "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "step": 1}),
+                "context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
+            },
+               "optional":{
+                    "custom_model_name":("STRING", {"forceInput": True,}), #适合自定义model
+                }, 
+        }
+
+    RETURN_TYPES = ("STRING","STRING","STRING",)
+    RETURN_NAMES = ("text","messages","session_history",)
+    FUNCTION = "generate_contextual_text"
+    CATEGORY = "♾️Mixlab/GPT"
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,False,)
+
+    
+    def generate_contextual_text(self,
+                                 api_key,
+                                 prompt, 
+                                 system_content,
+                                model, 
+                                seed,context_size,custom_model_name=None):
+
+        if custom_model_name!=None:
+            model=custom_model_name
+
+        api_url="https://api.siliconflow.cn/v1"
+        
+        # 把系统信息和初始信息添加到会话历史中
+        if system_content:
+            self.system_content=system_content
+            # self.session_history=[]
+            # self.session_history.append({"role": "system", "content": system_content})
+        
+        # 
+        client = openai_client(api_key,api_url)  # 使用 ChatGPT  的接口
+        # print('using ChatGPT interface',api_key,api_url)
+
+        # 把用户的提示添加到会话历史中
+        # 调用API时传递整个会话历史
+
+        def crop_list_tail(lst, size):
+            if size >= len(lst):
+                return lst
+            elif size==0:
+                return []
+            else:
+                return lst[-size:]
+            
+        session_history=crop_list_tail(self.session_history,context_size)
+
+        messages=[{"role": "system", "content": self.system_content}]+session_history+[{"role": "user", "content": prompt}]
+
+        response_content = chat(client,model,messages)
+        
+        self.session_history=self.session_history+[{"role": "user", "content": prompt}]+[{'role':'assistant',"content":response_content}]
+
+        return (response_content,json.dumps(messages, indent=4),json.dumps(self.session_history, indent=4),)
+
+
+
 
 class ShowTextForGPT:
     @classmethod
@@ -484,3 +686,35 @@ class TextSplitByDelimiter:
         arr= arr[start_index:start_index + max_count * (skip_every+1):(skip_every+1)]
 
         return (arr,)
+
+
+class JsonRepair:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                 "json_string":("STRING", {"forceInput": True,}), 
+            }
+        }
+
+    INPUT_IS_LIST = False
+    RETURN_TYPES = ("STRING",)
+    FUNCTION = "run"
+    # OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (False,)
+
+    CATEGORY = "♾️Mixlab/GPT"
+
+    def run(self, json_string):
+
+        json_string=extract_json_strings(json_string)
+        # print(json_string)
+        good_json_string = repair_json(json_string)
+
+        # 将 JSON 字符串解析为 Python 对象
+        data = json.loads(good_json_string)
+
+        # 将 Python 对象转换回 JSON 字符串，确保中文字符不被转义
+        json_str_with_chinese = json.dumps(data, ensure_ascii=False)
+
+        return (json_str_with_chinese,)

nodes/ClipInterrogator.py

@@ -79,7 +79,7 @@ def get_clip_interrogator_path():
 
 cache_path=get_clip_interrogator_path()
 
-caption_model_path=os.path.join(cache_path, "Salesforce/blip-image-captioning-base")
+caption_model_path=os.path.join(cache_path, "Salesforce","blip-image-captioning-base")
 if not os.path.exists(caption_model_path):
     print(f"## clip_interrogator_model not found: {caption_model_path}, pls download from https://huggingface.co/Salesforce/blip-image-captioning-base")
     caption_model_path='Salesforce/blip-image-captioning-base'

nodes/ImageNode.py

@@ -1,6 +1,7 @@
 import numpy as np
 import requests
 import torch
+import torchvision.transforms.v2 as T
 # from PIL import Image, ImageDraw
 from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence, ImageFont
 from PIL.PngImagePlugin import PngInfo
@@ -14,8 +15,8 @@ import cv2
 import string
 import math,glob
 from .Watcher import FolderWatcher
-import hashlib
 
+from itertools import product
 
 
 # 将PIL图片转换为OpenCV格式
@@ -28,142 +29,105 @@ def opencv_to_pil(image):
     pil_image = Image.fromarray(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
     return pil_image
 
+# 列出目录下面的所有文件
+def get_files_with_extension(directory, extensions):
+    file_list = []
+    # 确保extensions参数是一个list，即使只有一个元素
+    if not isinstance(extensions, (tuple, list)):
+        extensions = [extensions]
+    for root, dirs, files in os.walk(directory):
+        # print(f"Files at {root}: {files}")  # 确认files是一个字符串列表
+        for file in files:
+            # 检查文件是否以任何一个提供的扩展名结尾
+            if any(file.endswith(ext) for ext in extensions):
+                # 直接将文件名添加到列表中
+                file_list.append(file)
+    return file_list
+	
+def composite_images(foreground, background, mask, is_multiply_blend=False, position="overall", scale=0.25):
+    width, height = foreground.size
+    bg_image = background
+    bwidth, bheight = bg_image.size
 
-def composite_images(foreground, background, mask,is_multiply_blend=False,position="overall"):
-    width,height=foreground.size
- 
-    bg_image=background
+    scale=max(scale,1/bwidth)
+    scale=max(scale,1/bheight)
 
-    bwidth,bheight=bg_image.size
+    def determine_scale_option(width, height):
+        return 'height' if height > width else 'width'
 
-    # 按z-index排序
-    if position=="overall":
+    if position == "overall":
         layer = {
-                "x":0,
-                "y":0,
-                "width":bwidth,
-                "height":bheight,
-                "z_index":88,
-                "scale_option":'overall',
-                "image":foreground,
-                "mask":mask
+            "x": 0,
+            "y": 0,
+            "width": bwidth,
+            "height": bheight,
+            "z_index": 88,
+            "scale_option": 'overall',
+            "image": foreground,
+            "mask": mask
         }
+    else:
+        scale_option = determine_scale_option(width, height)
+        if scale_option == 'height':
+            scale = int(bheight * scale) / height
+        else:
+            scale = int(bwidth * scale) / width
 
-    elif position=='center_bottom':
-        
-        scale = int(bwidth*0.25) / width
+        new_width = int(width * scale)
         new_height = int(height * scale)
 
-        layer = {
-                "x":int(bwidth*0.75*0.5),
-                "y":bheight-new_height-24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
-        }
-
-    elif position=='right_bottom':
-        
-        scale = int(bwidth*0.25) / width
-        new_height = int(height * scale)
+        if position == 'center_bottom':
+            x_position = int((bwidth - new_width) * 0.5)
+            y_position = bheight - new_height - 24
+        elif position == 'right_bottom':
+            x_position = bwidth - new_width - 24
+            y_position = bheight - new_height - 24
+        elif position == 'center_top':
+            x_position = int((bwidth - new_width) * 0.5)
+            y_position = 24
+        elif position == 'right_top':
+            x_position = bwidth - new_width - 24
+            y_position = 24
+        elif position == 'left_top':
+            x_position = 24
+            y_position = 24
+        elif position == 'left_bottom':
+            x_position = 24
+            y_position = bheight - new_height - 24
+        elif position == 'center_center':
+            x_position = int((bwidth - new_width) * 0.5)
+            y_position = int((bheight - new_height) * 0.5)
 
         layer = {
-                "x":bwidth-int(bwidth*0.25)-24,
-                "y":bheight-new_height-24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
+            "x": x_position,
+            "y": y_position,
+            "width": new_width,
+            "height": new_height,
+            "z_index": 88,
+            "scale_option": scale_option,
+            "image": foreground,
+            "mask": mask
         }
 
+    layer_image = layer['image']
+    layer_mask = layer['mask']
 
-    elif position=='center_top':
-        
-        scale = int(bwidth*0.25) / width
-        new_height = int(height * scale)
+    bg_image = merge_images(bg_image,
+                            layer_image,
+                            layer_mask,
+                            layer['x'],
+                            layer['y'],
+                            layer['width'],
+                            layer['height'],
+                            layer['scale_option'],
+                            is_multiply_blend)
 
-        layer = {
-                "x":int( bwidth*0.75*0.5),
-                "y":24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
-        }
+    bg_image = bg_image.convert('RGB')
 
-    elif position=='right_top':
-        
-        scale = int(bwidth*0.25) / width
-        new_height = int(height * scale)
-
-        layer = {
-                "x":bwidth-int(bwidth*0.25)-24,
-                "y":24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
-        }
-    elif position=='left_top':
-        
-        scale = int(bwidth*0.25) / width
-        new_height = int(height * scale)
-
-        layer = {
-                "x":24,
-                "y":24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
-        }
-    elif position=='left_bottom':
-        
-        scale = int(bwidth*0.25) / width
-        new_height = int(height * scale)
-
-        layer = {
-                "x":24,
-                "y":bheight-new_height-24,
-                "width":int(bwidth*0.25),
-                "height":int(bheight*0.25),
-                "z_index":88,
-                "scale_option":'width',
-                "image":foreground,
-                "mask":mask
-        }
-                
-    # width, height = bg_image.size
-
-    layer_image=layer['image']
-    layer_mask=layer['mask']
-    
-    bg_image=merge_images(bg_image,
-                        layer_image,
-                        layer_mask,
-                        layer['x'],
-                        layer['y'],
-                        layer['width'],
-                        layer['height'],
-                        layer['scale_option'],
-                        is_multiply_blend )
-                   
-    bg_image=bg_image.convert('RGB')
-    
     return bg_image
 
 
+
 def count_files_in_directory(directory):
     file_count = 0
     for _, _, files in os.walk(directory):
@@ -200,7 +164,8 @@ class AnyType(str):
 any_type = AnyType("*")
 
 
-FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
+FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),"..","assets","fonts"))
+
 
 MAX_RESOLUTION=8192
 
@@ -802,85 +767,78 @@ def multiply_blend(image1, image2):
 
 # cv2.imwrite('result.jpg', result)
 
+# 使用gpt4o优化代码
+# 为了消除图像合并时出现的灰色描边，可以使用以下方法：
+# 调整透明度：确保透明像素不会引入不需要的颜色。
+# 预处理图像：在缩放图像之前，可以先将图像的边缘进行预处理，例如扩展边缘颜色，减少抗锯齿带来的过渡效果。
 
-def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,is_multiply_blend=False):
+def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option, is_multiply_blend=False):
     # 打开底图
     bg_image = bg_image.convert("RGBA")
 
     # 打开图层
     layer_image = layer_image.convert("RGBA")
-    # layer_image = layer_image.resize((width, height))
-
+   
     # 根据缩放选项调整图像大小
     if scale_option == "height":
         # 按照高度比例缩放
         original_width, original_height = layer_image.size
         scale = height / original_height
         new_width = int(original_width * scale)
-        layer_image = layer_image.resize((new_width, height))
+        layer_image = layer_image.resize((new_width, height), Image.NEAREST)
     elif scale_option == "width":
         # 按照宽度比例缩放
         original_width, original_height = layer_image.size
         scale = width / original_width
         new_height = int(original_height * scale)
-        layer_image = layer_image.resize((width, new_height))
+        layer_image = layer_image.resize((width, new_height), Image.NEAREST)
     elif scale_option == "overall":
         # 整体缩放
-        layer_image = layer_image.resize((width, height))
-
+        layer_image = layer_image.resize((width, height), Image.NEAREST)
     elif scale_option == "longest":
         original_width, original_height = layer_image.size
         if original_width > original_height:
-            new_width=width
+            new_width = width
             scale = width / original_width
             new_height = int(original_height * scale)
-            x=0
-            y=int((height-new_height)*0.5)
+            x = 0
+            y = int((height - new_height) * 0.5)
         else:
-            new_height=height
+            new_height = height
             scale = height / original_height
             new_width = int(original_height * scale)
-            x=int((width-new_width)*0.5)
-            y=0
-    # elif side == "shortest":
-    #         if width < height:
-    #              
-    #         else:
-    #              
-    
+            x = int((width - new_width) * 0.5)
+            y = 0
 
     # 调整mask的大小
     nw, nh = layer_image.size
-    mask = mask.resize((nw, nh))
+    mask = mask.resize((nw, nh), Image.NEAREST)
 
-    # # 分离出a通道
-    # r, g, b, alpha = layer_image.split()
-    # alpha = ImageOps.invert(alpha)
-    # # 创建一个新的RGB图像
-    # new_rgb_image = Image.new("RGB", layer_image.size)
-    # # 将透明通道粘贴到新的RGB图像上
-    # new_rgb_image.paste(layer_image, (0, 0), mask=alpha)
-   
-    # new_rgb_image.paste(layer_image, (x, y), mask=mask)
-    # mask=new_rgb_image.convert('L')
-    # mask = ImageOps.invert(mask)
+    # 预处理图像边缘以减少灰色描边
+    layer_image = layer_image.filter(ImageFilter.SMOOTH)
 
     if is_multiply_blend:
-        bg_image_white=Image.new("RGB", bg_image.size,(255, 255, 255))
+        bg_image_white = Image.new("RGB", bg_image.size, (255, 255, 255))
 
         bg_image_white.paste(layer_image, (x, y), mask=mask)
-        bg_image=multiply_blend(bg_image_white,bg_image)
-        bg_image=bg_image.convert("RGBA")
+        bg_image = multiply_blend(bg_image_white, bg_image)
+        bg_image = bg_image.convert("RGBA")
     else:
-        transparent_img = Image.new("RGBA",layer_image.size, (255, 255, 255, 0))
-        transparent_img.paste(layer_image,(0, 0), mask)
-        # transparent_img.save('test.png')
-        bg_image.paste(transparent_img, (x, y), transparent_img)
+        transparent_img = Image.new("RGBA", layer_image.size, (255, 255, 255, 0))
+        # 调整透明度处理
+        for i in range(transparent_img.size[0]):
+            for j in range(transparent_img.size[1]):
+                r, g, b, a = transparent_img.getpixel((i, j))
+                if a > 0:
+                    transparent_img.putpixel((i, j), (r, g, b, 255))
 
+        transparent_img.paste(layer_image, (0, 0), mask)
+        bg_image.paste(transparent_img, (x, y), transparent_img)
 
     # 输出合成后的图片
     return bg_image
 
+#MixCopilot
 
 def resize_2(img):
     # 检查图像的高度是否是2的倍数，如果不是，则调整高度
@@ -954,53 +912,13 @@ def resize_image(layer_image, scale_option, width, height,color="white"):
     return layer_image
 
 
-
-
-
-
-# def generate_text_image(text_list, font_path, font_size, text_color, vertical=True, spacing=0):
-#     # Load Chinese font
-#     font = ImageFont.truetype(font_path, font_size)
-
-#     # Calculate image size based on the number of characters and orientation
-#     if vertical:
-#         width = font_size + 100
-#         height = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
-#     else:
-#         width = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
-#         height = font_size + 100
-
-#     # Create a blank image
-#     image = Image.new('RGBA', (width, height), (255, 255, 255,0))
-#     draw = ImageDraw.Draw(image)
-
-#     # Draw text
-#     if vertical:
-#         for i, char in enumerate(text_list):
-#             char_position = (50, 50 + i * font_size)
-#             draw.text(char_position, char, font=font, fill=text_color)
-#     else:
-#         for i, char in enumerate(text_list):
-#             char_position = (50 + i * (font_size + spacing), 50)
-#             draw.text(char_position, char, font=font, fill=text_color)
-
-#     # Save the image
-#     # image.save(output_image_path)
-
-#     # 分离alpha通道
-#     alpha_channel = image.split()[3]
-
-#     # 创建一个只有alpha通道的新图像
-#     alpha_image = Image.new('L', image.size)
-#     alpha_image.putdata(alpha_channel.getdata())
-
-#     image=image.convert('RGB')
-
-#     return (image,alpha_image)
-def generate_text_image(text, font_path, font_size, text_color, vertical=True, stroke=False, stroke_color=(0, 0, 0), stroke_width=1, spacing=0):
+def generate_text_image(text, font_path, font_size, text_color, vertical=True, stroke=False, stroke_color=(0, 0, 0), stroke_width=1, spacing=0, line_spacing=0,padding=4):
     # Split text into lines based on line breaks
     lines = text.split("\n")
 
+    # Load font
+    font = ImageFont.truetype(font_path, font_size)
+
     # 1. Determine layout direction
     if vertical:
         layout = "vertical"
@@ -1009,49 +927,54 @@ def generate_text_image(text, font_path, font_size, text_color, vertical=True, s
 
     # 2. Calculate absolute coordinates for each character
     char_coordinates = []
-    if layout == "vertical":
-        x = 0
-        y = 0
-        for i in range(len(lines)):
-            line = lines[i]
-            for char in line:
-                char_coordinates.append((x, y))
-                y += font_size + spacing
-            x += font_size + spacing
-            y = 0
-    else:
-        x = 0
-        y = 0
-        for line in lines:
-            for char in line:
-                char_coordinates.append((x, y))
-                x += font_size + spacing
-            y += font_size + spacing
-            x = 0
+    x, y = padding, padding
+    max_width, max_height = 0, 0
 
-    # 3. Calculate image width and height
     if layout == "vertical":
-        width = (len(lines) * (font_size + spacing)) - spacing
-        height = ((len(max(lines, key=len)) + 1) * (font_size + spacing)) + spacing
+        for line in lines:
+            max_char_width = max(font.getsize(char)[0] for char in line)
+            for char in line:
+                char_width, char_height = font.getsize(char)
+                char_coordinates.append((x, y))
+                y += char_height + spacing
+                max_height = max(max_height, y + padding)
+            x += max_char_width + line_spacing
+            y = padding
+        max_width = x
+        total_line_width = sum(font.getsize(line)[1] for line in lines)
+        total_spacing = line_spacing * (len(lines) - 1)
+        # 确保左边和右边的padding都被计入max_width
+        max_width = total_line_width + total_spacing + padding * 2
     else:
-        width = (len(max(lines, key=len)) * (font_size + spacing)) - spacing
-        height = ((len(lines) - 1) * (font_size + spacing)) + font_size
+        for line in lines:
+            line_width, line_height = font.getsize(line)
+            for char in line:
+                char_width, char_height = font.getsize(char)
+                char_coordinates.append((x, y))
+                x += char_width + spacing
+                max_width = max(max_width, x + padding)
+            y += line_height + line_spacing
+            x = padding
+        # max_height = y
+        total_line_heights = sum(font.getsize(line)[1] for line in lines)
+        total_spacing = line_spacing * (len(lines) - 1)
+        # 确保顶部和底部的padding都被计入max_height
+        max_height = total_line_heights + total_spacing + padding * 2
+
+    # 3. Create image with calculated width and height
+    image = Image.new('RGBA', (max_width, max_height), (255, 255, 255, 0))
+    draw = ImageDraw.Draw(image)
 
     # 4. Draw each character on the image
-    image = Image.new('RGBA', (width, height), (255, 255, 255, 0))
-    draw = ImageDraw.Draw(image)
-    font = ImageFont.truetype(font_path, font_size)
-
     index = 0
-    for i, line in enumerate(lines):
-        for j, char in enumerate(line):
+    for line in lines:
+        for char in line:
             x, y = char_coordinates[index]
-            
             if stroke:
-                draw.text((x-stroke_width, y), char, font=font, fill=stroke_color)
-                draw.text((x+stroke_width, y), char, font=font, fill=stroke_color)
-                draw.text((x, y-stroke_width), char, font=font, fill=stroke_color)
-                draw.text((x, y+stroke_width), char, font=font, fill=stroke_color)
+                draw.text((x-stroke_width, y), char, font=font, fill=text_color)
+                draw.text((x+stroke_width, y), char, font=font, fill=text_color)
+                draw.text((x, y-stroke_width), char, font=font, fill=text_color)
+                draw.text((x, y+stroke_width), char, font=font, fill=text_color)
             
             draw.text((x, y), char, font=font, fill=text_color)
             index += 1
@@ -1376,6 +1299,9 @@ class LoadImages_:
             image=pil2tensor(image)
             ims.append(image)
 
+        if len(ims)==0:
+            image1 = Image.new('RGB', (512, 512), color='black')
+            return (pil2tensor(image1),)
         image1 = ims[0]
         for image2 in ims[1:]:
             if image1.shape[1:] != image2.shape[1:]:
@@ -1578,7 +1504,7 @@ class ImageCropByAlpha:
 
 
 
-
+# get_files_with_extension(FONT_PATH,'.ttf')
 
 class TextImage:
     @classmethod
@@ -1586,18 +1512,32 @@ class TextImage:
         return {"required": { 
             
                     "text": ("STRING",{"multiline": True,"default": "龍馬精神迎新歲","dynamicPrompts": False}),
-                    "font_path": ("STRING",{"multiline": False,"default": FONT_PATH,"dynamicPrompts": False}),
+                    "font": (get_files_with_extension(FONT_PATH,['.ttf','.otf']),),#后缀为 ttf
                     "font_size": ("INT",{
                                 "default":100, 
-                                "min": 100, #Minimum value
-                                "max": 1000, #Maximum value
+                                "min": 1, #Minimum value
+                                "max": 10000000, #Maximum value
                                 "step": 1, #Slider's step
                                 "display": "number" # Cosmetic only: display as "number" or "slider"
                                 }), 
                     "spacing": ("INT",{
                                 "default":12, 
-                                "min": -200, #Minimum value
-                                "max": 200, #Maximum value
+                                "min": -2000000000, #Minimum value
+                                "max": 2000000000, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "line_spacing": ("INT",{
+                                "default":12, 
+                                "min": -2000000000, #Minimum value
+                                "max": 2000000000, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "padding": ("INT",{
+                                "default":8, 
+                                "min": 0, #Minimum value
+                                "max": 2000000000, #Maximum value
                                 "step": 1, #Slider's step
                                 "display": "number" # Cosmetic only: display as "number" or "slider"
                                 }), 
@@ -1608,7 +1548,7 @@ class TextImage:
                 }
     
     RETURN_TYPES = ("IMAGE","MASK",)
-    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+    RETURN_NAMES = ("image","mask",)
 
     FUNCTION = "run"
 
@@ -1617,11 +1557,14 @@ class TextImage:
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,False,)
 
-    def run(self,text,font_path,font_size,spacing,text_color,vertical,stroke):
+    def run(self,text,font,font_size,spacing,line_spacing,padding,text_color,vertical,stroke):
         
-        # text_list=list(text)
+        font_path=os.path.join(FONT_PATH,font)
+
+        if text=="":
+            text=" "
         # stroke=False, stroke_color=(0, 0, 0), stroke_width=1, spacing=0
-        img,mask=generate_text_image(text,font_path,font_size,text_color,vertical,stroke,(0, 0, 0),1,spacing)
+        img,mask=generate_text_image(text,font_path,font_size,text_color,vertical,stroke,(0, 0, 0),1,spacing,line_spacing,padding)
         
         img=pil2tensor(img)
         mask=pil2tensor(mask)
@@ -1655,7 +1598,7 @@ class LoadImagesFromURL:
 
     def run(self,url,seed=0):
         global urls_image
-        print(urls_image)
+        # print(urls_image)
         def filter_http_urls(urls):
             filtered_urls = []
             for url in urls.split('\n'):
@@ -1854,10 +1797,16 @@ class CompositeImages:
                     "mask":("MASK",),
                     "background": ("IMAGE",),
                     },
-             "optional":{
-                           
+             "optional":{ 
                   "is_multiply_blend":  ("BOOLEAN", {"default": False}),
-                  "position":  (['overall',"center_bottom","center_top","right_bottom","left_bottom","right_top","left_top"],),    
+                  "position":  (['overall',"center_center","left_bottom","center_bottom","right_bottom","left_top","center_top","right_top"],),
+                   "scale": ("FLOAT",{
+                                "default":0.35, 
+                                "min": 0.01, #Minimum value
+                                "max": 1, #Maximum value
+                                "step": 0.01, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                            }), 
                     }
                 }
     
@@ -1870,15 +1819,30 @@ class CompositeImages:
 
     # OUTPUT_IS_LIST = (True,)
 
-    def run(self, foreground,mask,background,is_multiply_blend,position):
-        foreground= tensor2pil(foreground)
-        mask= tensor2pil(mask)
-        background= tensor2pil(background)
-        res=composite_images(foreground,background,mask,is_multiply_blend,position)
+    # def run(self, foreground,mask,background,is_multiply_blend,position,scale):
+    #     foreground= tensor2pil(foreground)
+    #     mask= tensor2pil(mask)
+    #     background= tensor2pil(background)
+    #     res=composite_images(foreground,background,mask,is_multiply_blend,position,scale)
         
-        return (pil2tensor(res),)
+    #     return (pil2tensor(res),)
 
+    def run(self, foreground,mask,background, is_multiply_blend, position, scale):
+        results = []
+        f1=[]
+        for fg, mask in zip(foreground, mask ):
+            f1.append([fg,mask])
+        for f, bg in product(f1, background):
+            [fg,mask]=f
+            fg_pil = tensor2pil(fg)
+            mask_pil = tensor2pil(mask)
+            bg_pil = tensor2pil(bg)
+            res = composite_images(fg_pil, bg_pil, mask_pil, is_multiply_blend, position, scale)
+            results.append(pil2tensor(res))
+        
+        output_image = torch.cat(results, dim=0)
 
+        return (output_image,)
 
 
 class EmptyLayer:
@@ -3207,3 +3171,52 @@ class SaveImageToLocal:
             counter += 1
 
         return ()
+
+
+class ImageBatchToList_:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {"image_batch": ("IMAGE",), }}
+
+    RETURN_TYPES = ("IMAGE",)
+    RETURN_NAMES = ("image_list",)
+    OUTPUT_IS_LIST = (True,)
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Image"
+
+    def run(self, image_batch):
+        images = [image_batch[i:i + 1, ...] for i in range(image_batch.shape[0])]
+        return (images, )
+
+class ImageListToBatch_:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "images": ("IMAGE",),
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "run"
+    INPUT_IS_LIST = True
+    CATEGORY = "♾️Mixlab/Image"
+
+    def run(self, images):
+        shape = images[0].shape[1:3]
+        out = []
+
+        for i in range(len(images)):
+            img = images[i].permute([0,3,1,2])
+            if images[i].shape[1:3] != shape:
+                transforms = T.Compose([
+                    T.CenterCrop(min(img.shape[2], img.shape[3])),
+                    T.Resize((shape[0], shape[1]), interpolation=T.InterpolationMode.BICUBIC),
+                ])
+                img = transforms(img)
+            out.append(img.permute([0,2,3,1]))
+
+        out = torch.cat(out, dim=0)
+
+        return (out,)

nodes/Lama.py

@@ -85,8 +85,6 @@ class LaMaInpainting:
             "image": ("IMAGE",),
             "mask": ("MASK",),
                              },
-
-            
                 }
     
     RETURN_TYPES = ("IMAGE",)

nodes/P5.py

@@ -0,0 +1,104 @@
+import torch
+import numpy as np
+from PIL import Image,ImageSequence,ImageOps
+import base64
+import io
+import comfy.utils
+import folder_paths
+import node_helpers
+
+
+# Tensor to PIL
+def tensor2pil(image):
+    return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
+
+# Convert PIL to Tensor
+def pil2tensor(image):
+    return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
+
+def load_image( image):
+    image_path = folder_paths.get_annotated_filepath(image)
+        
+    img = node_helpers.pillow(Image.open, image_path)
+        
+    output_images = []
+    output_masks = []
+    w, h = None, None
+
+    excluded_formats = ['MPO']
+        
+    for i in ImageSequence.Iterator(img):
+        i = node_helpers.pillow(ImageOps.exif_transpose, i)
+
+        if i.mode == 'I':
+            i = i.point(lambda i: i * (1 / 255))
+        image = i.convert("RGB")
+
+        if len(output_images) == 0:
+            w = image.size[0]
+            h = image.size[1]
+            
+        if image.size[0] != w or image.size[1] != h:
+            continue
+            
+        image = np.array(image).astype(np.float32) / 255.0
+        image = torch.from_numpy(image)[None,]
+        if 'A' in i.getbands():
+            mask = np.array(i.getchannel('A')).astype(np.float32) / 255.0
+            mask = 1. - torch.from_numpy(mask)
+        else:
+            mask = torch.zeros((64,64), dtype=torch.float32, device="cpu")
+        output_images.append(image)
+        output_masks.append(mask.unsqueeze(0))
+
+    if len(output_images) > 1 and img.format not in excluded_formats:
+        output_image = torch.cat(output_images, dim=0)
+        output_mask = torch.cat(output_masks, dim=0)
+    else:
+        output_image = output_images[0]
+        output_mask = output_masks[0]
+
+    return (output_image, output_mask)
+
+          
+
+class P5Input:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "frames":("IMAGEBASE64",), 
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE",)
+    RETURN_NAMES = ("frames",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Input"
+
+    OUTPUT_NODE = True
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self, frames):
+        ims=[]
+        for im in frames['images']:
+            # print(im)
+            if 'type' in im and (not f"[{im['type']}]" in im['name']):
+                im['name']=im['name']+" "+f"[{im['type']}]"
+                
+            output_image, output_mask = load_image(im['name'])
+            ims.append(output_image)
+
+        if len(ims)==0:
+            image1 = Image.new('RGB', (512, 512), color='black')
+            return (pil2tensor(image1),)
+        image1 = ims[0]
+        for image2 in ims[1:]:
+            if image1.shape[1:] != image2.shape[1:]:
+                image2 = comfy.utils.common_upscale(image2.movedim(-1, 1), image1.shape[2], image1.shape[1], "bilinear", "center").movedim(1, -1)
+            image1 = torch.cat((image1, image2), dim=0)
+
+        # 用于节点提示:p5节点提示有多少帧
+        return {"ui": {"_info": [len(frames['images'])]}, "result": (image1,)}

nodes/ScreenShareNode.py

@@ -90,7 +90,7 @@ class ScreenShareNode:
                 } }
     
     RETURN_TYPES = ('IMAGE','STRING','FLOAT',"INT")
-    RETURN_NAMES = ("IMAGE","PROMPT","FLOAT","INT")
+    RETURN_NAMES = ("current frame (image)","prompt","denoise (float)","seed (int)")
     FUNCTION = "run"
 
     CATEGORY = "♾️Mixlab/Screen"
@@ -109,7 +109,7 @@ class FloatingVideo:
     @classmethod
     def INPUT_TYPES(s):
         return { "required":{
-            "images": ("IMAGE",)
+            "image": ("IMAGE",)
         }, }
     
     # RETURN_TYPES = ('IMAGE','MASK')
@@ -124,16 +124,16 @@ class FloatingVideo:
     # OUTPUT_IS_LIST = (False,False,)
   
     # 运行的函数
-    def run(self,images):
+    def run(self,image):
        
         results = list()
     
-        for image in images:
-            image=tensor2pil(image)
+        for im in image:
+            im=tensor2pil(im)
             # image_base64 = base64.b64encode(image.tobytes())
 
             buffered = BytesIO()
-            image.save(buffered, format="JPEG")
+            im.save(buffered, format="JPEG")
             image_base64 = base64.b64encode(buffered.getvalue()).decode("utf-8")
 
             results.append(image_base64)

nodes/Utils.py

@@ -133,7 +133,7 @@ def get_font_files(directory):
 
     return font_files
 
-r_directory = os.path.join(os.path.dirname(__file__), '../assets/')
+r_directory = os.path.join(os.path.dirname(__file__), '..','assets','/')
 
 font_files = get_font_files(r_directory)
 # print(font_files)
@@ -181,6 +181,28 @@ class ColorInput:
         return (h,r,g,b,a,)
 
 
+class KeyInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "key":("KEY",), 
+                             },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    RETURN_NAMES = ("key",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Input"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,key):
+        return (key,)
+
+
 
 class FontInput:
     @classmethod
@@ -566,7 +588,7 @@ class AppInfo:
                              },
 
                 "optional":{
-                    "IMAGE": ("IMAGE",),
+                    "image": ("IMAGE",),
                     "description":("STRING",{"multiline": True,"default": "","dynamicPrompts": False}),
                     "version":("INT", {
                         "default": 1, 
@@ -594,12 +616,12 @@ class AppInfo:
     INPUT_IS_LIST = True
     # OUTPUT_IS_LIST = (True,)
 
-    def run(self,name,input_ids,output_ids,IMAGE,description,version,share_prefix,link,category,auto_save):
+    def run(self,name,input_ids,output_ids,image,description,version,share_prefix,link,category,auto_save):
         name=name[0]
         
         im=None
-        if IMAGE:
-            im=IMAGE[0][0]
+        if image:
+            im=image[0][0]
             #TODO batch 的方式需要处理
             im=create_temp_file(im)
         # image [img,] img[batch,w,h,a] 列表里面是batch，

nodes/Video.py

@@ -17,9 +17,128 @@ import folder_paths
 from comfy.k_diffusion.utils import FolderOfImages
 from comfy.utils import common_upscale
 
+import torchaudio
+import base64
+
+import mimetypes
 
 
 
+def get_frames(frame_count, frames, revert=False):
+    if not revert:
+        if frame_count <= len(frames):
+            return frames[:frame_count]
+        else:
+            return [frames[i % len(frames)] for i in range(frame_count)]
+    else:
+        extended_frames = frames + frames[-2:0:-1]  # 正向加反向中间部分
+        if frame_count <= len(extended_frames):
+            return extended_frames[:frame_count]
+        else:
+            return [extended_frames[i % len(extended_frames)] for i in range(frame_count)]
+
+# # 示例用法
+# frames = ["frame1", "frame2", "frame3"]
+# frame_count = 2
+
+# result = get_frames(frame_count, frames, revert=False)
+# print(result)  # 输出: ['frame1', 'frame2', 'frame3', 'frame1', 'frame2', 'frame3', 'frame1']
+
+# result = get_frames(frame_count, frames, revert=True)
+# print(result)  # 输出: ['frame1', 'frame2', 'frame3', 'frame2', 'frame1', 'frame2', 'frame3']
+
+
+
+
+def get_mime_type(file_path):
+    # 获取文件的 MIME 类型
+    mime_type, _ = mimetypes.guess_type(file_path)
+    
+    # 如果无法猜测类型，返回默认类型
+    if mime_type is None:
+        return 'application/octet-stream'
+    
+    return mime_type
+# import subprocess
+# from imageio_ffmpeg import get_ffmpeg_exe
+
+
+def save_audio_base64s_to_file(base64_audios, output_folder, file_name):
+    # Ensure the output folder exists
+    if not os.path.exists(output_folder):
+        os.makedirs(output_folder)
+
+    decoded_audios=[]
+    for a in base64_audios:
+    
+        # If the base64 string contains a header, remove it
+        if ',' in a:
+            a = a.split(',')[1]
+        
+        # 解码 base64 数据
+        a=base64.b64decode(a)
+        decoded_audios.append(a)
+
+    # 拼接音频数据
+    combined_audio = b''.join(decoded_audios)
+    
+    # Create the full file path
+    file_path = os.path.join(output_folder, file_name)
+    
+    # Write the decoded audio to the file
+    with open(file_path, 'wb') as audio_file:
+        audio_file.write(combined_audio)
+    
+    return file_path
+
+# Example usage
+# base64_audio = "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAIlYAAESsAAACABAAZGF0YQAAAAA="
+# output_folder = "audio_files"
+# file_name = "output.wav"
+
+# file_path = save_audio_base64_to_file(base64_audio, output_folder, file_name)
+# print(f"Audio saved to: {file_path}")
+
+# 写一个python文件，用来 判断文件夹内命名为 所有chat_tts开头的文件数量（chat_tts_00001），并输出新的编号
+def get_new_counter(full_output_folder, filename_prefix):
+    # 获取目录中的所有文件
+    files = os.listdir(full_output_folder)
+    
+    # 过滤出以 filename_prefix 开头并且后续部分为数字的文件
+    filtered_files = []
+    for f in files:
+        if f.startswith(filename_prefix):
+            # 去掉文件名中的前缀和后缀，只保留中间的数字部分
+            base_name = f[len(filename_prefix)+1:]
+            number_part = base_name.split('.')[0]  # 假设文件名中只有一个点，即扩展名
+            if number_part.isdigit():
+                filtered_files.append(int(number_part))
+
+    if not filtered_files:
+        return 1
+
+    # 获取最大的编号
+    max_number = max(filtered_files)
+    
+    # 新的编号
+    return max_number + 1
+
+def crop_audio(input_file, start_time, duration):
+    # Load the audio file
+    audio_tensor, sample_rate = torchaudio.load(input_file)
+    
+    # Convert start_time and duration from seconds to sample indices
+    start_sample = int(start_time * sample_rate)
+    end_sample = start_sample + int(duration * sample_rate)
+    
+    # Perform the slicing
+    cropped_audio_tensor = audio_tensor[:, start_sample:end_sample]
+    
+    # Save the cropped audio to a new file
+    torchaudio.save(input_file, cropped_audio_tensor, sample_rate)
+    
+    return input_file
+
 def generate_folder_name(directory,video_path):
     # Get the directory and filename from the video path
     _, filename = os.path.split(video_path)
@@ -60,6 +179,9 @@ def split_video(video_path, video_segment_frames, transition_frames, output_dir)
         
         # 打印当前片段的起始帧和结束帧
         print(f"Segment {i+1}: Start Frame {start_frame}, End Frame {end_frame}")
+
+        if end_frame<start_frame:
+            break
         
         # 保存当前片段为一个视频文件
         segment_video_path = f"{output_dir}/segment_{i+1}.avi"
@@ -68,6 +190,7 @@ def split_video(video_path, video_segment_frames, transition_frames, output_dir)
         segment_video = cv2.VideoWriter(segment_video_path, fourcc, fps, (int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH)),
                                                               int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))))
         
+
         for frame_num in range(start_frame, end_frame):
             ret, frame = video_capture.read()
             if ret:
@@ -101,6 +224,25 @@ if ffmpeg_path is None:
     except:
         print("ffmpeg could not be found. Outputs that require it have been disabled")
 
+
+def combine_audio_video(audio_path, video_path, output_path):
+   
+    command = [
+        ffmpeg_path,
+        '-i', video_path,
+        '-i', audio_path,
+        '-c:v', 'copy',
+        '-c:a', 'aac',
+        '-shortest',
+        output_path
+    ]
+    
+    subprocess.run(command, check=True)
+    return output_path
+
+
+
+
 # Tensor to PIL
 def tensor2pil(image):
     return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
@@ -262,7 +404,7 @@ class LoadVideoAndSegment:
                     files.append(f)
         return {"required": {
                     "video": (sorted(files), {"video_upload": True}),
-                     "video_segment_frames": ("INT", {"default": 10, "min": 1, "step": 1}),
+                     "video_segment_frames": ("INT", {"default": 10, "min": -1, "step": 1}),
                      "transition_frames": ("INT", {"default": 0, "min": 0, "step": 1}), 
                      },}
 
@@ -332,63 +474,6 @@ class LoadVideoAndSegment:
         
         video_path = folder_paths.get_annotated_filepath(video)
         
-        # check if video is a gif - will need to use cv fallback to read frames
-        # use cv fallback if ffmpeg not installed or gif
-        # if ffmpeg_path is None:
-        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
-        # otherwise, continue with ffmpeg
-        
-        # args_dummy = [ffmpeg_path, "-i", video_path, "-f", "null", "-"]
-        # try:
-        #     with subprocess.Popen(args_dummy, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) as proc:
-        #         for line in proc.stderr.readlines():
-        #             match = re.search(", ([1-9]|\\d{2,})x(\\d+)",line.decode('utf-8'))
-        #             if match is not None:
-        #                 size = [int(match.group(1)), int(match.group(2))]
-        #                 break
-        # except Exception as e:
-        #     print(f"Retrying with opencv due to ffmpeg error: {e}")
-        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
-        # args_all_frames = [ffmpeg_path, "-i", video_path, "-v", "error",
-        #                      "-pix_fmt", "rgb24"]
-
-        # vfilters = []
-       
-        # if skip_first_frames > 0:
-        #     vfilters.append(f"select=gt(n\\,{skip_first_frames-1})")
-        # if frame_load_cap > 0:
-        #     vfilters.append(f"select=gt({frame_load_cap}\\,n)")
-        # #manually calculate aspect ratio to ensure reads remain aligned
-        
-        # if len(vfilters) > 0:
-        #     args_all_frames += ["-vf", ",".join(vfilters)]
-
-        # args_all_frames += ["-f", "rawvideo", "-"]
-        # images = []
-        # try:
-        #     with subprocess.Popen(args_all_frames, stdout=subprocess.PIPE) as proc:
-        #         #Manually buffer enough bytes for an image
-        #         bpi = size[0]*size[1]*3
-        #         current_bytes = bytearray(bpi)
-        #         current_offset=0
-        #         while True:
-        #             bytes_read = proc.stdout.read(bpi - current_offset)
-        #             if bytes_read is None:#sleep to wait for more data
-        #                 time.sleep(.2)
-        #                 continue
-        #             if len(bytes_read) == 0:#EOF
-        #                 break
-        #             current_bytes[current_offset:len(bytes_read)] = bytes_read
-        #             current_offset+=len(bytes_read)
-        #             if current_offset == bpi:
-        #                 images.append(np.array(current_bytes, dtype=np.float32).reshape(size[1], size[0], 3) / 255.0)
-        #                 current_offset = 0
-        # except Exception as e:
-        #     print(f"Retrying with opencv due to ffmpeg error: {e}")
-        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
-
-        # imgs=split_list(images,video_segment_frames,transition_frames)
-
         # temp path 
         tp=folder_paths.get_temp_directory()
         basename = os.path.basename(video_path)  # 获取文件名
@@ -396,15 +481,22 @@ class LoadVideoAndSegment:
 
         folder_path = create_folder(tp,name_without_extension)
         
- 
-        # 导出的数据
-        scenes_video,total_frames,fps=split_video(video_path,video_segment_frames,
-                                 transition_frames,folder_path)
+        if video_segment_frames==-1:
+            # 不切割视频
+            scenes_video=[video_path]
+            # 读取视频文件
+            video_capture = cv2.VideoCapture(video_path)
+            
+            # 获取视频的总帧数和帧率
+            total_frames = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
+            fps = video_capture.get(cv2.CAP_PROP_FPS)
+
+        else:
+            # 导出的数据
+            scenes_video,total_frames,fps=split_video(video_path,video_segment_frames,
+                                    transition_frames,folder_path)
       
 
-        # imgs=[torch.from_numpy(np.stack(im)) for im in imgs]
-
-        # images = torch.from_numpy(np.stack(images))
 
         return (scenes_video,len(scenes_video), total_frames,fps,)
 
@@ -422,7 +514,113 @@ class LoadVideoAndSegment:
             return "Invalid image file: {}".format(video)
 
         return True
-    
+
+
+
+class LoadAndCombinedAudio_:
+    @classmethod
+    def INPUT_TYPES(s):
+       
+        return {"required": {
+                    "audios": ("AUDIOBASE64",),
+                    "start_time": ("FLOAT" , {"default": 0, "min": 0, "max": 10000000, "step": 0.01}),
+                    "duration": ("FLOAT" , {"default": 10, "min": -1, "max": 10000000, "step": 0.01}),
+                     },
+                }
+
+    CATEGORY = "♾️Mixlab/Audio"
+
+    RETURN_TYPES = ("STRING","AUDIO",)
+    RETURN_NAMES = ("audio_file_path","audio",)
+    FUNCTION = "run"
+
+    def run(self,audios, start_time, duration):
+        output_dir = folder_paths.get_output_directory()
+        counter=get_new_counter(output_dir,'audio_')
+
+        audio_file_name = f"audio_{counter:05}.wav"
+
+        audio_file=save_audio_base64s_to_file(audios['base64'],output_dir,audio_file_name)
+        # duration == -1 则不裁切
+        if duration > -1:
+            crop_audio(audio_file, start_time, duration)
+
+        waveform, sample_rate = torchaudio.load(audio_file)
+        audio = {
+            "filename": audio_file_name,
+            "subfolder": "",
+            "type": "output",
+            "audio_path":audio_file,
+            "waveform": waveform.unsqueeze(0), 
+            "sample_rate": sample_rate}
+
+        return (audio_file,audio ,)
+
+class CombineAudioVideo:
+    @classmethod
+    def INPUT_TYPES(s):
+       
+        return {"required": {
+                     "video": ("SCENE_VIDEO",),
+                     "audio": ("AUDIO", ), 
+                     },
+                }
+
+    CATEGORY = "♾️Mixlab/Video"
+
+    OUTPUT_NODE = True
+    FUNCTION = "run" 
+    RETURN_TYPES = ("SCENE_VIDEO",)
+    RETURN_NAMES = ("SCENE_VIDEO",)
+
+    def run(self,video, audio):
+
+        output_dir = folder_paths.get_output_directory()
+
+        # 判断是否是 Tensor 类型
+        is_tensor = not isinstance(audio, dict)
+        # print('#判断是否是 Tensor 类型',is_tensor,audio)
+        if not is_tensor and 'waveform' in audio and 'sample_rate' in audio:
+            # {'waveform': tensor([], size=(1, 1, 0)), 'sample_rate': 44100}
+            is_tensor=True
+
+        if "audio_path" in audio:
+            is_tensor=False
+            audio_file_path=audio["audio_path"]
+
+        if is_tensor:
+            filename_prefix="audio_tmp"
+            full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
+                filename_prefix, 
+                folder_paths.get_temp_directory())
+            
+            filename_with_batch_num = filename.replace("%batch_num%", str(1))
+            file = f"{filename_with_batch_num}_{counter:05}_.wav"
+            
+            audio_file_path=os.path.join(full_output_folder, file)
+
+            torchaudio.save(audio_file_path, audio['waveform'].squeeze(0), audio["sample_rate"])
+            
+        # 获取文件名和扩展名
+        base, ext = os.path.splitext(video)
+        counter=get_new_counter(output_dir,'video_final_')
+
+        v_file = f"video_final_{counter:05}{ext}"
+        
+        v_file_path=os.path.join(output_dir, v_file)
+
+        combine_audio_video(audio_file_path,video,v_file_path)
+
+        previews = [
+            {
+                "filename": v_file,
+                "subfolder": "",
+                "type": "output",
+                "format": get_mime_type(v_file),
+            }
+        ]
+
+        return {"ui": {"gifs": previews},"result":(v_file_path,)}
 
 # The code is based on ComfyUI-VideoHelperSuite modification.
 class VideoCombine_Adv:
@@ -454,7 +652,8 @@ class VideoCombine_Adv:
             },
         }
 
-    RETURN_TYPES = ()
+    RETURN_TYPES = ("SCENE_VIDEO",)
+    RETURN_NAMES = ("scenes_video",)
     OUTPUT_NODE = True
     CATEGORY = "♾️Mixlab/Video"
     FUNCTION = "run"
@@ -623,7 +822,7 @@ class VideoCombine_Adv:
                 "format": format,
             }
         ]
-        return {"ui": {"gifs": previews}}
+        return {"ui": {"gifs": previews},"result":(file_path,)}
 
 
 class VAEEncodeForInpaint_Frames:
@@ -690,4 +889,113 @@ class VAEEncodeForInpaint_Frames:
             result.append({"samples":t, "noise_mask": (mask_erosion[:,:,:x,:y].round())})
 
 
-        return (result, )
+        return (result, )
+    
+
+
+class GenerateFramesByCount:
+    @classmethod
+    def INPUT_TYPES(cls):
+        
+        return {"required": {
+                    "frames": ('IMAGE',),
+                    "frame_count": ("INT", {"default": 72, "min": 1, "step": 1}), 
+                    "revert" :("BOOLEAN", {"default": True},),
+                     },}
+
+    RETURN_TYPES = ('IMAGE',)
+    RETURN_NAMES = ("frames",)
+
+    FUNCTION = "r"
+    CATEGORY = "♾️Mixlab/Video"
+    # INPUT_IS_LIST = True
+
+    def r(self, frames, frame_count, revert):
+        
+        image_list = [frames[i:i + 1, ...] for i in range(frames.shape[0])]
+
+        image_list=get_frames(frame_count,image_list,revert)
+
+        images = torch.cat(image_list, dim=0)
+
+        return (images,)
+
+
+class scenesNode_:
+    @classmethod
+    def INPUT_TYPES(cls):
+        
+        return {"required": {
+                    "scenes_video": ('SCENE_VIDEO',),
+                    "index": ("INT", {"default": 0, "min": 0, "step": 1}),
+                     
+                     },}
+
+    RETURN_TYPES = ('IMAGE','INT',)
+    RETURN_NAMES = ("video frames (batch)","count",)
+    # OUTPUT_IS_LIST = (False,)
+
+    FUNCTION = "run"
+    CATEGORY = "♾️Mixlab/Video"
+    INPUT_IS_LIST = True
+
+    def load_video_cv_fallback(self, video, frame_load_cap, skip_first_frames):
+        # print('#video',video)
+        try:
+            video_cap = cv2.VideoCapture(video)
+            if not video_cap.isOpened():
+                raise ValueError(f"{video} could not be loaded with cv fallback.")
+            # set video_cap to look at start_index frame
+            images = []
+            total_frame_count = 0
+            frames_added = 0
+            base_frame_time = 1/video_cap.get(cv2.CAP_PROP_FPS)
+            
+            target_frame_time = base_frame_time
+
+            time_offset=0.0
+            while video_cap.isOpened():
+                if time_offset < target_frame_time:
+                    is_returned, frame = video_cap.read()
+                    # if didn't return frame, video has ended
+                    if not is_returned:
+                        break
+                    time_offset += base_frame_time
+                if time_offset < target_frame_time:
+                    continue
+                time_offset -= target_frame_time
+                # if not at start_index, skip doing anything with frame
+                total_frame_count += 1
+                if total_frame_count <= skip_first_frames:
+                    continue
+                # TODO: do whatever operations need to happen, like force_size, etc
+
+                # opencv loads images in BGR format (yuck), so need to convert to RGB for ComfyUI use
+                # follow up: can videos ever have an alpha channel?
+                frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                # convert frame to comfyui's expected format (taken from comfy's load image code)
+                image = Image.fromarray(frame)
+                image = ImageOps.exif_transpose(image)
+                image = np.array(image, dtype=np.float32) / 255.0
+                image = torch.from_numpy(image)[None,]
+                images.append(image)
+                frames_added += 1
+                # if cap exists and we've reached it, stop processing frames
+                if frame_load_cap > 0 and frames_added >= frame_load_cap:
+                    break
+        finally:
+            video_cap.release()
+        
+        images = torch.cat(images, dim=0)
+       
+        return (images, frames_added,)
+
+    def run(self, scenes_video,index):
+        print('#scenes_video',index,scenes_video)
+        index=index[0]
+        if len(scenes_video) > index:
+            vp=scenes_video[index]
+        else:
+            vp=scenes_video[-1]
+        
+        return self.load_video_cv_fallback(vp,0,0)

nodes/edit_mask.py

@@ -0,0 +1,172 @@
+import torch
+from PIL import Image, ImageOps, ImageSequence, ImageFile
+from PIL.PngImagePlugin import PngInfo
+
+import numpy as np
+import os
+import folder_paths 
+import node_helpers
+import hashlib
+
+# Tensor to PIL
+def tensor2pil(image):
+    return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
+
+# tensor 取hash值
+def tensor_to_hash(tensor):
+    # 将 Tensor 转换为 NumPy 数组
+    np_array = tensor.cpu().numpy()
+    
+    # 将 NumPy 数组转换为字节数据
+    byte_data = np_array.tobytes()
+    
+    # 计算哈希值
+    hash_value = hashlib.md5(byte_data).hexdigest()
+    
+    return hash_value
+
+
+def create_temp_file(image):
+    output_dir = folder_paths.get_temp_directory()
+
+    (
+            full_output_folder,
+            filename,
+            counter,
+            subfolder,
+            _,
+        ) = folder_paths.get_save_image_path('material', output_dir)
+
+    
+    image=tensor2pil(image)
+ 
+    image_file = f"{filename}_{counter:05}.png"
+     
+    image_path=os.path.join(full_output_folder, image_file)
+
+    image.save(image_path,compress_level=4)
+
+    return (image_path,[{
+    "filename": image_file,
+    "subfolder": subfolder,
+    "type": "temp"
+    }])
+
+
+# image - tensor - 文件路径
+# loadImage的方法（ 文件路径 - image-mask ）
+class EditMask:
+
+    def __init__(self):
+        self.image_id = None
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required":
+                    {"image": ("IMAGE",), # 表示一个张量
+                     
+                     },
+
+                    "optional":{
+                            "image_update": ("IMAGE_FILE",)
+                        },
+                   
+                }
+
+    CATEGORY = "♾️Mixlab/Mask"
+
+    RETURN_TYPES = ("IMAGE", "MASK")
+    RETURN_NAMES = ("image", "mask")
+
+    FUNCTION = "edit"
+
+    OUTPUT_NODE = True
+
+    def edit(self, image,image_update=None):
+
+        # 根据image输入来判断是否是新的图片
+        if self.image_id==None:
+            self.image_id=tensor_to_hash(image)
+            image_update=None
+        else:
+            image_id=tensor_to_hash(image)
+            if image_id!=self.image_id:
+                image_update=None
+                self.image_id=image_id
+
+
+        image_path=None
+        # print('#image_update',self.image_id,image_update)
+        if image_update==None:
+            print('--')
+        else:
+            if 'images' in image_update:
+                images=image_update['images']
+                filename=images[0]['filename']
+                subfolder=images[0]['subfolder']
+                type=images[0]['type']
+                name, base_dir=folder_paths.annotated_filepath(filename)
+                if type.endswith("output"):
+                    base_dir = folder_paths.get_output_directory() 
+                elif type.endswith("input"):
+                    base_dir = folder_paths.get_input_directory() 
+                elif type.endswith("temp"):
+                    base_dir = folder_paths.get_temp_directory() 
+                #base_dir = folder_paths.get_input_directory()  
+                # print(base_dir,subfolder, name)
+                image_path = os.path.join(base_dir,subfolder, name)
+        
+        if image_path==None:
+            image_path,images=create_temp_file(image)
+
+        print('#image_path',os.path.exists(image_path),image_path)
+        # image_path = folder_paths.get_annotated_filepath(image) #文件名
+        
+        if not os.path.exists(image_path):
+            image_path,images=create_temp_file(image)
+
+
+        img = node_helpers.pillow(Image.open, image_path)
+        
+        output_images = []
+        output_masks = []
+        w, h = None, None
+
+        excluded_formats = ['MPO']
+        
+        for i in ImageSequence.Iterator(img):
+            i = node_helpers.pillow(ImageOps.exif_transpose, i)
+
+            if i.mode == 'I':
+                i = i.point(lambda i: i * (1 / 255))
+            image = i.convert("RGB")
+
+            if len(output_images) == 0:
+                w = image.size[0]
+                h = image.size[1]
+            
+            if image.size[0] != w or image.size[1] != h:
+                continue
+            
+            image = np.array(image).astype(np.float32) / 255.0
+            image = torch.from_numpy(image)[None,]
+            if 'A' in i.getbands():
+                mask = np.array(i.getchannel('A')).astype(np.float32) / 255.0
+                mask = 1. - torch.from_numpy(mask)
+            else:
+                # 尺寸不对，需要按照image来
+                mask = torch.zeros((h, w), dtype=torch.float32, device="cpu")
+
+            output_images.append(image)
+            output_masks.append(mask.unsqueeze(0))
+
+        if len(output_images) > 1 and img.format not in excluded_formats:
+            output_image = torch.cat(output_images, dim=0)
+            output_mask = torch.cat(output_masks, dim=0)
+        else:
+            output_image = output_images[0]
+            output_mask = output_masks[0]
+
+        return {"ui":{"images": images},"result": (output_image, output_mask)}
+
+        # return (output_image, output_mask)

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "comfyui-mixlab-nodes"
 description = "3D, ScreenShareNode & FloatingVideoNode, SpeechRecognition & SpeechSynthesis, GPT, LoadImagesFromLocal, Layers, Other Nodes, ..."
-version = "0.28.3"
+version = "0.35.2"
 license = "MIT"
 dependencies = ["numpy", "pyOpenSSL", "watchdog", "opencv-python-headless", "matplotlib", "openai", "simple-lama-inpainting", "clip-interrogator==0.6.0", "transformers>=4.36.0", "lark-parser", "imageio-ffmpeg", "rembg[gpu]", "omegaconf==2.3.0", "Pillow>=9.5.0", "einops==0.7.0", "trimesh>=4.0.5", "huggingface-hub", "scikit-image"]
 

requirements.txt

@@ -4,7 +4,7 @@ watchdog
 opencv-python-headless
 matplotlib
 openai
-simple-lama-inpainting
+# simple-lama-inpainting
 clip-interrogator==0.6.0
 transformers>=4.36.0
 lark-parser
@@ -15,4 +15,7 @@ Pillow>=9.5.0
 einops==0.7.0
 trimesh>=4.0.5
 huggingface-hub
-scikit-image
+scikit-image
+torchaudio
+soundfile>=0.12.1
+json-repair

web/ar.html

@@ -0,0 +1,22 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Mixlab AR</title>
+</head>
+
+<body>
+
+    <script type="module">
+
+        import { api } from "../../../scripts/api.js";
+        import Command from '/extensions/comfyui-mixlab-nodes/javascript/command.js'
+
+
+    </script>
+
+</body>
+
+</html>

web/javascript/app_mixlab.js

@@ -2,6 +2,9 @@ import { app } from '../../../scripts/app.js'
 import { $el } from '../../../scripts/ui.js'
 import { api } from '../../../scripts/api.js'
 
+import { td_bg } from './td_background.js'
+// console.log('td_bg', td_bg)
+
 //本机安装的插件节点全集
 window._nodesAll = null
 
@@ -185,6 +188,25 @@ async function extractInputAndOutputData (
         if (node.type == 'Color') {
         }
 
+        // 语音输入的支持
+        if (node.type == 'LoadAndCombinedAudio_') {
+          // if (
+          //   data[id].widgets_values &&
+          //   data[id].widgets_values[0] &&
+          //   data[id].widgets_values[0].base64 &&
+          //   data[id].widgets_values[0].base64.length > 0
+          // ) {
+          //   options.defaultBase64 = data[id].widgets_values[0].base64
+          // }
+
+          input[inputIds.indexOf(id)] = {
+            ...data[id],
+            title: node.title,
+            id,
+            options
+          }
+        }
+
         if (node.type === 'LoadImage') {
           // loadImage的mask支持
           let output = node.outputs.filter(ot => ot.type == 'MASK')[0]
@@ -235,7 +257,9 @@ async function extractInputAndOutputData (
         node.type === 'KSampler' ||
         node.type == 'SamplerCustom' ||
         node.type === 'ChinesePrompt_Mix' ||
-        node.type === 'Seed_'
+        node.type === 'Seed_'||
+        node.type==='SiliconflowLLM'||
+        node.type==='ChatGPTOpenAI'
       ) {
         // seed 的类型收集
         try {
@@ -397,11 +421,11 @@ async function save (json, download = false, showInfo = true) {
 
 function getInputsAndOutputs () {
   const inputs =
-      `LoadImage LoadImagesToBatch ImagesPrompt_ VHS_LoadVideo CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
+      `LoadImage LoadImagesToBatch ImagesPrompt_ LoadAndCombinedAudio_ LoadVideoAndSegment_ VHS_LoadVideo CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
         ' '
       ),
     outputs =
-      `SaveTripoSRMesh,PreviewImage,SaveImage,TransparentImage,ShowTextForGPT,VHS_VideoCombine,VideoCombine_Adv,Image Save,SaveImageAndMetadata_,ClipInterrogator`.split(
+      `SaveTripoSRMesh,PreviewImage,SaveImage,TransparentImage,ShowTextForGPT,CombineAudioVideo,VHS_VideoCombine,VideoCombine_Adv,Image Save,SaveImageAndMetadata_,ClipInterrogator`.split(
         ','
       )
 
@@ -445,19 +469,18 @@ app.registerExtension({
         const { input, output } = getInputsAndOutputs()
         input_ids.value = input.join('\n')
         output_ids.value = output.join('\n')
-
         const widget = {
           type: 'div',
           name: 'AppInfoRun',
           draw (ctx, node, widget_width, y, widget_height) {
             Object.assign(
               this.div.style,
-              get_position_style(
+              {...get_position_style(
                 ctx,
                 widget_width,
                 node.size[1] - widget_height,
                 node.size[1]
-              )
+              ),zIndex:1}
             )
           }
         }
@@ -503,6 +526,21 @@ app.registerExtension({
           }
         })
 
+        //td bg
+        const tdBG = document.createElement('button')
+        tdBG.innerText = 'Canvas Mode'
+        tdBG.style = style
+        tdBG.style.marginLeft = '12px'
+
+        tdBG.addEventListener('click', () => {
+          td_bg.toggle()
+          if (td_bg.running) {
+            tdBG.style.background = 'yellow'
+          } else {
+            tdBG.style.background = 'transparent'
+          }
+        })
+
         // author
         let author = document.createElement('div')
         // author.style=`display: flex`
@@ -659,6 +697,7 @@ app.registerExtension({
 
         btns.appendChild(btn)
         btns.appendChild(download)
+        btns.appendChild(tdBG)
 
         document.body.appendChild(widget.div)
         this.addCustomWidget(widget)
@@ -672,6 +711,7 @@ app.registerExtension({
         this.serialize_widgets = true //需要保存参数
 
         window._mixlab_app_json = null
+ 
       }
 
       const onExecuted = nodeType.prototype.onExecuted
@@ -687,9 +727,8 @@ app.registerExtension({
           }
 
           const div = this.widgets.filter(w => w.div)[0].div
-          Array.from(
-            div.querySelectorAll('button'),
-            b => (b.style.background = 'yellow')
+          Array.from(div.querySelectorAll('button'), b =>
+            b.innerText != 'Canvas Mode' ? (b.style.background = 'yellow') : ''
           )
         } catch (error) {}
       }

web/javascript/audio_mixlab.js

@@ -396,3 +396,218 @@ app.registerExtension({
     }
   }
 })
+
+// 上传音频转为base64
+async function uploadAndConvertAudio (file) {
+  if (!file) {
+    alert('Please select a WAV file.')
+    return
+  }
+
+  if (file.type !== 'audio/wav') {
+    alert('Only WAV files are supported.')
+    return
+  }
+
+  try {
+    const base64Audio = await readFileAsDataURL(file)
+    return base64Audio
+  } catch (error) {
+    console.error('Error reading file:', error)
+    alert('Error reading file.')
+  }
+}
+
+function readFileAsDataURL (file) {
+  return new Promise((resolve, reject) => {
+    const reader = new FileReader()
+
+    reader.onload = function (event) {
+      resolve(event.target.result)
+    }
+
+    reader.onerror = function (error) {
+      reject(error)
+    }
+
+    reader.readAsDataURL(file)
+  })
+}
+
+const createInputAudioForBatch = (base64, widget) => {
+  // Create an audio element
+  let audio = document.createElement('audio')
+  audio.src = base64
+  audio.controls = true
+  audio.style = 'width: 120px; display: block'
+
+  // Create a delete button
+  let deleteButton = document.createElement('button')
+  deleteButton.textContent = 'Delete'
+
+  deleteButton.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        margin-left: 10px;
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+  // Create a container for the audio and delete button
+  let container = document.createElement('div')
+  container.appendChild(audio)
+  container.appendChild(deleteButton)
+  container.style = `display: flex;margin-top: 12px;`
+
+  // Add event listener for the delete button
+  deleteButton.addEventListener('click', e => {
+    let newValue = []
+    let items = widget.value?.base64 || []
+    for (const v of items) {
+      if (v != base64) newValue.push(v)
+    }
+    widget.value.base64 = newValue
+    container.remove()
+  })
+
+  return container
+}
+
+app.registerExtension({
+  name: 'Mixlab.Comfy.LoadAndCombinedAudio_',
+  async getCustomWidgets (app) {
+    return {
+      AUDIOBASE64 (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          value: {
+            base64: []
+          }, // 不能[x,x,x]
+          type: inputData[0], // the type
+          name: inputName, // the name, slice
+          size: [128, 32], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 122] // a method to compute the current size of the widget
+          }
+          // serializeValue (nodeId, widgetIndex) {
+          //   return widget.value
+          // },
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'LoadAndCombinedAudio_') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        let audiosWidget = this.widgets.filter(w => w.name == 'audios')[0]
+
+        const widget = {
+          type: 'div',
+          name: 'audio_base64',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          },
+          serialize: false
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        let audioPreview = document.createElement('div')
+        let audiosDiv = document.createElement('div') //显示图片
+        audiosDiv.className = 'audios_preview'
+        audiosDiv.style = `width: calc(100% - 14px);
+        display: flex;
+        flex-wrap: wrap;
+        padding: 7px;    justify-content: space-between;
+        align-items: center;`
+
+        const btn = document.createElement('button')
+        btn.innerText = 'Upload Audio'
+
+        btn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        btn.addEventListener('click', e => {
+          e.preventDefault()
+          let inputAudio = document.createElement('input')
+          inputAudio.type = 'file'
+          inputAudio.accept = "audio/*"
+          inputAudio.style.display = 'none'
+          inputAudio.addEventListener('change', async e => {
+            e.preventDefault()
+            const file = e.target.files[0]
+            let base64 = await uploadAndConvertAudio(file)
+            if (!audiosWidget.value) audiosWidget.value = { base64: [] }
+            audiosWidget.value.base64.push(base64)
+
+            let a = createInputAudioForBatch(base64, audiosWidget)
+            audiosDiv.appendChild(a)
+          })
+
+          inputAudio.click()
+          inputAudio.remove()
+        })
+
+        widget.div.appendChild(audioPreview)
+        audioPreview.appendChild(audiosDiv)
+        audioPreview.appendChild(btn)
+        // audioPreview.appendChild(inputAudio)
+
+        this.addCustomWidget(widget)
+
+        // document.addEventListener('wheel', handleMouseWheel)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          try {
+            // document.removeEventListener('wheel', handleMouseWheel)
+          } catch (error) {
+            console.log(error)
+          }
+
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'LoadAndCombinedAudio_') {
+      // await sleep(0)
+      let audiosWidget = node.widgets.filter(w => w.name === 'audios')[0]
+      let audioPreview = node.widgets.filter(w => w.name == 'audio_base64')[0]
+
+      let pre = audioPreview.div.querySelector('.audios_preview')
+      for (const d of audiosWidget.value?.base64 || []) {
+        let im = createInputAudioForBatch(d, audiosWidget)
+        pre.appendChild(im)
+      }
+    }
+  }
+})

web/javascript/checkVersion_mixlab.js

@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
 const repoOwner = 'shadowcz007' // 替换为仓库的所有者
 const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
 
-const version = 'v0.28.3'
+const version = 'v0.35.1'
 
 fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
   .then(response => response.json())

web/javascript/command.js

@@ -0,0 +1,689 @@
+function get_url () {
+  // 如果有缓存记录
+  let hostUrl = localStorage.getItem('_hostUrl') || ''
+  if (hostUrl) {
+    return hostUrl
+  }
+  let api_host = `${window.location.hostname}:${window.location.port}`
+  let api_base = ''
+  let url = `${window.location.protocol}//${api_host}${api_base}`
+  return url
+}
+
+function getFilenameAndCategoryFromUrl (url) {
+  const queryString = url.split('?')[1]
+  if (!queryString) {
+    return {}
+  }
+
+  const params = new URLSearchParams(queryString)
+
+  const filename = params.get('filename')
+    ? decodeURIComponent(params.get('filename'))
+    : null
+  const category = params.get('category')
+    ? decodeURIComponent(params.get('category') || '')
+    : ''
+
+  return { category, filename }
+}
+
+async function get_my_app (category = '', filename = null) {
+  let url = get_url()
+  const res = await fetch(`${url}/mixlab/workflow`, {
+    method: 'POST',
+    mode: 'cors', // 允许跨域请求
+    headers: {
+      'Content-Type': 'application/json'
+    },
+    body: JSON.stringify({
+      task: 'my_app',
+      filename,
+      category
+    })
+  })
+  let result = await res.json()
+  let data = []
+  try {
+    for (const res of result.data) {
+      let { output, app } = res.data
+      if (app.filename)
+        data.push({
+          ...app,
+          data: output,
+          date: res.date
+        })
+    }
+  } catch (error) {}
+
+  return data
+}
+
+async function getAppInit () {
+  const { category, filename } = getFilenameAndCategoryFromUrl(
+    window.location.href
+  )
+  return await get_my_app(category, filename)
+}
+
+function success (isSuccess, btn, text) {
+  isSuccess ? (btn.innerText = 'success') : text
+  setTimeout(() => {
+    btn.innerText = text
+  }, 5000)
+}
+
+async function interrupt () {
+  try {
+    await fetch(`${get_url()}/interrupt`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: undefined
+    })
+  } catch (error) {
+    console.error(error)
+  }
+  return true
+}
+
+async function getQueue (clientId) {
+  try {
+    const res = await fetch(`${get_url()}/queue`)
+    const data = await res.json()
+    return {
+      // Running action uses a different endpoint for cancelling
+      Running: Array.from(data.queue_running, prompt => {
+        if (prompt[3].client_id === clientId) {
+          let prompt_id = prompt[1]
+          return {
+            prompt_id,
+            remove: () => interrupt()
+          }
+        }
+      }),
+      Pending: data.queue_pending.map(prompt => ({ prompt }))
+    }
+  } catch (error) {
+    console.error(error)
+    return { Running: [], Pending: [] }
+  }
+}
+
+// 请求历史数据
+async function getPromptResult (category) {
+  let url = get_url()
+  try {
+    const response = await fetch(`${url}/mixlab/prompt_result`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        action: 'all'
+      })
+    })
+
+    if (response.ok) {
+      const data = await response.json()
+      console.log('#getPromptResult:', category, data)
+
+      return data.result.filter(r => r.appInfo.category == category)
+      // 处理返回的数据
+    } else {
+      console.log('Error:', response.status)
+      // 处理错误情况
+    }
+  } catch (error) {
+    console.log('Error:', error)
+    // 处理异常情况
+  }
+}
+
+// 新的运行工作流的接口
+function queuePromptNew (filename, category, seed, input, client_id,apps=null) {
+  let url = get_url()
+  // var filename = "Text-to-Image_1.json", category = "";
+
+  // 随机seed
+  //  promptWorkflow = randomSeed(seed, promptWorkflow);
+  let d = { filename, category, seed, input, client_id }
+  if (apps) {
+    d.apps = apps
+  }
+
+  const data = JSON.stringify(d)
+  return new Promise((res, rej) => {
+    fetch(`${url}/mixlab/prompt`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: data
+    })
+      .then(response => {
+        if (!response.ok) {
+          // Handle HTTP error responses
+          if (response.status === 400) {
+            return response.json().then(errorData => {
+              // Process the error data
+              console.error('Error 400:', errorData)
+              alert(JSON.stringify(errorData, null, 2))
+              res(null)
+            })
+          }
+          throw new Error('Network response was not ok')
+        }
+        return response.json() // Process the response data
+      })
+      .then(data => {
+        // Handle the response data
+        console.log('Success:', data)
+        res(true)
+      })
+      .catch(error => {
+        // Handle fetch errors
+        console.error('Fetch error:', error)
+        res(null)
+      })
+  })
+}
+
+// 保存历史数据
+async function savePromptResult (data) {
+  let url = get_url()
+  try {
+    const response = await fetch(`${url}/mixlab/prompt_result`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        action: 'save',
+        data
+      })
+    })
+
+    if (response.ok) {
+      const res = await response.json()
+      console.log('Response:', res)
+      return res
+      // 处理返回的数据
+    } else {
+      console.log('Error:', response.status)
+      // 处理错误情况
+    }
+  } catch (error) {
+    console.log('Error:', error)
+    // 处理异常情况
+  }
+}
+
+async function uploadImage (blob, fileType = '.png', filename) {
+  const body = new FormData()
+  body.append(
+    'image',
+    new File([blob], (filename || new Date().getTime()) + fileType)
+  )
+
+  const url = get_url()
+
+  const resp = await fetch(`${url}/upload/image`, {
+    method: 'POST',
+    body
+  })
+
+  let data = await resp.json()
+  // console.log(data)
+  let { name, subfolder } = data
+  let src = `${url}/view?filename=${encodeURIComponent(
+    name
+  )}&type=input&subfolder=${subfolder}&rand=${Math.random()}`
+
+  return { url: src, name }
+}
+
+async function uploadMask (arrayBuffer, imgurl) {
+  const body = new FormData()
+  const filename = 'clipspace-mask-' + performance.now() + '.png'
+
+  let original_url = new URL(imgurl)
+
+  const original_ref = { filename: original_url.searchParams.get('filename') }
+
+  let original_subfolder = original_url.searchParams.get('subfolder')
+  if (original_subfolder) original_ref.subfolder = original_subfolder
+
+  let original_type = original_url.searchParams.get('type')
+  if (original_type) original_ref.type = original_type
+
+  body.append('image', arrayBuffer, filename)
+  body.append('original_ref', JSON.stringify(original_ref))
+  body.append('type', 'input')
+  body.append('subfolder', 'clipspace')
+
+  const url = get_url()
+
+  const resp = await fetch(`${url}/upload/mask`, {
+    method: 'POST',
+    body
+  })
+
+  // console.log(resp)
+  let data = await resp.json()
+  let { name, subfolder, type } = data
+  let src = `${url}/view?filename=${encodeURIComponent(
+    name
+  )}&type=${type}&subfolder=${subfolder}&rand=${Math.random()}`
+
+  return { url: src, name: 'clipspace/' + name }
+}
+
+const parseImageToBase64 = url => {
+  return new Promise((res, rej) => {
+    fetch(url)
+      .then(response => response.blob())
+      .then(blob => {
+        const reader = new FileReader()
+        reader.onloadend = () => {
+          const base64data = reader.result
+          res(base64data)
+          // 在这里可以将base64数据用于进一步处理或显示图片
+        }
+        reader.readAsDataURL(blob)
+      })
+      .catch(error => {
+        console.log('发生错误:', error)
+      })
+  })
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+function convertImageToBlackBasedOnAlpha (image) {
+  const canvas = document.createElement('canvas')
+  const ctx = canvas.getContext('2d')
+
+  // Draw the image onto the canvas
+  canvas.width = image.width
+  canvas.height = image.height
+  ctx.drawImage(image, 0, 0)
+
+  // Get the image data from the canvas
+  const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height)
+  const pixels = imageData.data
+
+  // Modify the RGB values based on the alpha channel
+  for (let i = 0; i < pixels.length; i += 4) {
+    const alpha = pixels[i + 3]
+    if (alpha !== 0) {
+      // Set non-transparent pixels to black
+      pixels[i] = 0 // Red
+      pixels[i + 1] = 0 // Green
+      pixels[i + 2] = 0 // Blue
+    }
+  }
+
+  // Put the modified image data back onto the canvas
+  ctx.putImageData(imageData, 0, 0)
+
+  // Convert the modified canvas to base64 data URL
+  const base64ImageData = canvas.toDataURL('image/png') // Replace 'png' with your desired image format
+
+  return base64ImageData
+}
+
+const blobToBase64 = blob => {
+  return new Promise((res, rej) => {
+    const reader = new FileReader()
+    reader.onloadend = () => {
+      const base64data = reader.result
+      res(base64data)
+      // 在这里可以将base64数据用于进一步处理或显示图片
+    }
+    reader.readAsDataURL(blob)
+  })
+}
+
+function base64ToBlob (base64) {
+  // 去除base64编码中的前缀
+  const base64WithoutPrefix = base64.replace(/^data:image\/\w+;base64,/, '')
+
+  // 将base64编码转换为字节数组
+  const byteCharacters = atob(base64WithoutPrefix)
+
+  // 创建一个存储字节数组的数组
+  const byteArrays = []
+
+  // 将字节数组放入数组中
+  for (let offset = 0; offset < byteCharacters.length; offset += 1024) {
+    const slice = byteCharacters.slice(offset, offset + 1024)
+
+    const byteNumbers = new Array(slice.length)
+    for (let i = 0; i < slice.length; i++) {
+      byteNumbers[i] = slice.charCodeAt(i)
+    }
+
+    const byteArray = new Uint8Array(byteNumbers)
+    byteArrays.push(byteArray)
+  }
+
+  // 创建blob对象
+  const blob = new Blob(byteArrays, { type: 'image/png' }) // 根据实际情况设置MIME类型
+
+  return blob
+}
+
+async function calculateImageHash (blob) {
+  const buffer = await blob.arrayBuffer()
+  const hashBuffer = await crypto.subtle.digest('SHA-256', buffer)
+  const hashArray = Array.from(new Uint8Array(hashBuffer))
+  const hashHex = hashArray
+    .map(byte => byte.toString(16).padStart(2, '0'))
+    .join('')
+  return hashHex
+}
+
+// 获取 rembg 模型
+async function get_rembg_models () {
+  try {
+    const response = await fetch(`${get_url()}/mixlab/folder_paths`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        type: 'rembg'
+      })
+    })
+
+    const data = await response.json()
+    // console.log(data)
+    return data.names
+  } catch (error) {
+    console.error(error)
+  }
+}
+
+//自动抠图
+async function run_rembg (model, base64) {
+  try {
+    const response = await fetch(`${get_url()}/mixlab/rembg`, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        model,
+        base64
+      })
+    })
+
+    const data = await response.json()
+    // console.log(data)
+    return data.data
+  } catch (error) {
+    console.error(error)
+  }
+}
+
+function copyHtmlWithImagesToClipboard (data, cb) {
+  // 创建一个临时div元素
+  const tempDiv = document.createElement('div')
+
+  // 将HTML字符串赋值给div的innerHTML属性
+  tempDiv.innerHTML = data
+
+  // 获取div中的所有图像元素
+  const images = tempDiv.getElementsByTagName('img')
+
+  // 遍历图像元素，并将图像数据转换为Base64编码
+  for (let i = 0; i < images.length; i++) {
+    const image = images[i]
+    const canvas = document.createElement('canvas')
+    const context = canvas.getContext('2d')
+
+    // 设置canvas尺寸与图像尺寸相同
+    canvas.width = image.width
+    canvas.height = image.height
+
+    // 在canvas上绘制图像
+    context.drawImage(image, 0, 0)
+
+    // 将canvas转换为Base64编码
+    const imageData = canvas.toDataURL()
+
+    // 将Base64编码替换图像元素的src属性
+    image.src = imageData
+  }
+
+  let richText = tempDiv.innerHTML
+
+  // 创建一个新的Blob对象，并将富文本字符串作为数据传递进去
+  const blob = new Blob([richText], { type: 'text/html' })
+
+  // 创建一个ClipboardItem对象，并将Blob对象添加到其中
+  const clipboardItem = new ClipboardItem({ 'text/html': blob })
+
+  // 使用Clipboard API将内容复制到剪贴板
+  navigator.clipboard
+    .write([clipboardItem])
+    .then(() => {
+      console.log('富文本已成功复制到剪贴板')
+      tempDiv.remove()
+      if (cb) cb(true)
+    })
+    .catch(error => {
+      console.error('复制到剪贴板失败:', error)
+      tempDiv.remove()
+      if (cb) cb(false)
+    })
+}
+
+function copyImagesToClipboard (html, cb) {
+  const tempDiv = document.createElement('div')
+  tempDiv.innerHTML = html
+  const images = tempDiv.querySelectorAll('img')
+  const promises = Array.from(images).map(image => {
+    return new Promise(resolve => {
+      const img = new Image()
+      img.src = image.src
+      img.onload = () => {
+        const canvas = document.createElement('canvas')
+        const context = canvas.getContext('2d')
+        canvas.width = img.width
+        canvas.height = img.height
+        context.drawImage(img, 0, 0)
+        canvas.toBlob(blob => {
+          const clipboardItem = new ClipboardItem({ 'image/png': blob })
+          navigator.clipboard
+            .write([clipboardItem])
+            .then(() => {
+              resolve()
+              tempDiv.remove()
+              if (cb) cb(true)
+            })
+            .catch(error => {
+              reject(error)
+              tempDiv.remove()
+              if (cb) cb(false)
+            })
+        })
+      }
+    })
+  })
+  Promise.all([...promises])
+    .then(() => {
+      console.log('所有图片已成功复制到剪贴板')
+      if (cb) cb(true)
+      tempDiv.remove()
+    })
+    .catch(error => {
+      console.error('复制到剪贴板失败:', error)
+      if (cb) cb(false)
+      tempDiv.remove()
+    })
+}
+
+function copyTextToClipboard (html, cb) {
+  const tempDiv = document.createElement('div')
+  tempDiv.innerHTML = html
+
+  const text = tempDiv.innerText
+  const textData = new ClipboardItem({
+    'text/plain': new Blob([text], { type: 'text/plain' })
+  })
+
+  navigator.clipboard
+    .write([textData])
+    .then(() => {
+      console.log('所有文本已成功复制到剪贴板', text)
+      if (cb) cb(true)
+      tempDiv.remove()
+    })
+    .catch(error => {
+      console.error('复制到剪贴板失败:', error)
+      if (cb) cb(false)
+      tempDiv.remove()
+    })
+}
+
+// ComfyUI\web\extensions\core\dynamicPrompts.js
+// 官方实现修改
+// Allows for simple dynamic prompt replacement
+// Inputs in the format {a|b} will have a random value of a or b chosen when the prompt is queued.
+
+/*
+ * Strips C-style line and block comments from a string
+ */
+function dynamicPrompts (prompt) {
+  prompt = prompt.replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, '')
+  while (
+    prompt.replace('\\{', '').includes('{') &&
+    prompt.replace('\\}', '').includes('}')
+  ) {
+    const startIndex = prompt.replace('\\{', '00').indexOf('{')
+    const endIndex = prompt.replace('\\}', '00').indexOf('}')
+
+    const optionsString = prompt.substring(startIndex + 1, endIndex)
+    const options = optionsString.split('|')
+
+    const randomIndex = Math.floor(Math.random() * options.length)
+    const randomOption = options[randomIndex]
+
+    prompt =
+      prompt.substring(0, startIndex) +
+      randomOption +
+      prompt.substring(endIndex + 1)
+  }
+  return prompt
+}
+
+// 遍历所有组合,语法同 动态提示
+function generateAllCombinations (prompt) {
+  prompt = prompt.replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, '')
+
+  // Helper function to get all combinations
+  function getAllCombinations (parts) {
+    if (parts.length === 0) return ['']
+    const [firstPart, ...restParts] = parts
+    const restCombinations = getAllCombinations(restParts)
+    const allCombinations = []
+
+    firstPart.forEach(option => {
+      restCombinations.forEach(combination => {
+        allCombinations.push(option + combination)
+      })
+    })
+
+    return allCombinations
+  }
+
+  // Split prompt into static parts and dynamic parts
+  let parts = []
+  let startIndex = 0
+
+  while (
+    prompt.replace('\\{', '').includes('{') &&
+    prompt.replace('\\}', '').includes('}')
+  ) {
+    startIndex = prompt.replace('\\{', '00').indexOf('{')
+    const endIndex = prompt.replace('\\}', '00').indexOf('}')
+    const staticPart = prompt.substring(0, startIndex)
+    const optionsString = prompt.substring(startIndex + 1, endIndex)
+    const options = optionsString.split('|')
+
+    parts.push([staticPart])
+    parts.push(options)
+
+    prompt = prompt.substring(endIndex + 1)
+  }
+
+  // Add the remaining static part
+  parts.push([prompt])
+
+  // Get all combinations
+  const combinations = getAllCombinations(parts)
+
+  return combinations
+}
+
+const _textNodes = [
+    'TextInput_',
+    'CLIPTextEncode',
+    'PromptSimplification',
+    'ChinesePrompt_Mix'
+  ],
+  _loraNodes = ['CheckpointLoaderSimple', 'LoraLoader'],
+  _numberNodes = ['FloatSlider', 'IntNumber'],
+  _slideNodes = ['PromptSlide'],
+  _imageNodes = [
+    'LoadImage',
+    'VHS_LoadVideo',
+    'ImagesPrompt_',
+    'LoadImagesToBatch'
+  ],
+  _colorNodes = ['Color'],
+  _audioNodes = ['LoadAndCombinedAudio_']
+
+export default {
+  get_url,
+  get_my_app,
+  getAppInit,
+  getFilenameAndCategoryFromUrl,
+  success,
+  interrupt,
+  getQueue,
+  queuePromptNew,
+  savePromptResult,
+  uploadImage,
+  uploadMask,
+  run_rembg,
+  get_rembg_models,
+  parseImageToBase64,
+  createImage,
+  convertImageToBlackBasedOnAlpha,
+  blobToBase64,
+  base64ToBlob,
+  calculateImageHash,
+  copyHtmlWithImagesToClipboard,
+  copyImagesToClipboard,
+  copyTextToClipboard,
+  dynamicPrompts,
+  generateAllCombinations,
+
+  _textNodes,
+  _loraNodes,
+  _numberNodes,
+  _slideNodes,
+  _imageNodes,
+  _colorNodes,
+  _audioNodes
+}

web/javascript/gpt_mixlab.js

@@ -1,205 +1,5 @@
 import { app } from '../../../scripts/app.js'
-// import { api } from '../../../scripts/api.js'
 import { ComfyWidgets } from '../../../scripts/widgets.js'
-import { $el } from '../../../scripts/ui.js'
-
-async function getConfig () {
-  let api_host = `${window.location.hostname}:${window.location.port}`
-  let api_base = ''
-  let url = `${window.location.protocol}//${api_host}${api_base}`
-
-  const res = await fetch(`${url}/mixlab`, {
-    method: 'POST'
-  })
-  return await res.json()
-}
-
-function get_position_style (ctx, widget_width, y, node_height) {
-  const MARGIN = 4 // the margin around the html element
-
-  /* Create a transform that deals with all the scrolling and zooming */
-  const elRect = ctx.canvas.getBoundingClientRect()
-  const transform = new DOMMatrix()
-    .scaleSelf(
-      elRect.width / ctx.canvas.width,
-      elRect.height / ctx.canvas.height
-    )
-    .multiplySelf(ctx.getTransform())
-    .translateSelf(MARGIN, MARGIN + y)
-
-  return {
-    transformOrigin: '0 0',
-    transform: transform,
-    left: `0`,
-    top: `0`,
-    cursor: 'pointer',
-    position: 'absolute',
-    maxWidth: `${widget_width - MARGIN * 2}px`,
-    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
-    width: `${widget_width - MARGIN * 2}px`,
-    // height: `${node_height * 0.3 - MARGIN * 2}px`,
-    // background: '#EEEEEE',
-    display: 'flex',
-    flexDirection: 'column',
-    // alignItems: 'center',
-    justifyContent: 'space-around'
-  }
-}
-
-const getLocalData = key => {
-  let data = {}
-  try {
-    data = JSON.parse(localStorage.getItem(key)) || {}
-  } catch (error) {
-    return {}
-  }
-  return data
-}
-
-app.registerExtension({
-  name: 'Mixlab.GPT.ChatGPTOpenAI',
-  async getCustomWidgets (app) {
-    return {
-      KEY (node, inputName, inputData, app) {
-        // console.log('##inputData', inputData)
-        const widget = {
-          type: inputData[0], // the type, CHEESE
-          name: inputName, // the name, slice
-          size: [128, 32], // a default size
-          draw (ctx, node, width, y) {},
-          computeSize (...args) {
-            return [128, 32] // a method to compute the current size of the widget
-          },
-          async serializeValue (nodeId, widgetIndex) {
-            let data = getLocalData('_mixlab_api_key')
-            return data[node.id] || 'by Mixlab'
-          }
-        }
-        //  widget.something = something;          // maybe adds stuff to it
-        node.addCustomWidget(widget) // adds it to the node
-        return widget // and returns it.
-      },
-      URL (node, inputName, inputData, app) {
-        // console.log('node', inputName, inputData[0])
-        const widget = {
-          type: inputData[0], // the type, CHEESE
-          name: inputName, // the name, slice
-          size: [128, 32], // a default size
-          draw (ctx, node, width, y) {
-            // a method to draw the widget (ctx is a CanvasRenderingContext2D)
-          },
-          computeSize (...args) {
-            return [128, 32] // a method to compute the current size of the widget
-          },
-          async serializeValue (nodeId, widgetIndex) {
-            let data = getLocalData('_mixlab_api_url')
-            return data[node.id] || 'https://api.openai.com/v1'
-          }
-        }
-        //  widget.something = something;          // maybe adds stuff to it
-        node.addCustomWidget(widget) // adds it to the node
-        return widget // and returns it.
-      }
-    }
-  },
-
-  async beforeRegisterNodeDef (nodeType, nodeData, app) {
-    if (nodeType.comfyClass == 'ChatGPTOpenAI') {
-      const orig_nodeCreated = nodeType.prototype.onNodeCreated
-      nodeType.prototype.onNodeCreated = function () {
-        orig_nodeCreated?.apply(this, arguments)
-
-        const api_key = this.widgets.filter(w => w.name == 'api_key')[0]
-        const api_url = this.widgets.filter(w => w.name == 'api_url')[0]
-
-        console.log('ChatGPTOpenAI nodeData', this.widgets)
-
-        const widget = {
-          type: 'div',
-          name: 'chatgptdiv',
-          draw (ctx, node, widget_width, y, widget_height) {
-            Object.assign(
-              this.div.style,
-              get_position_style(ctx, widget_width, api_key.y, node.size[1])
-            )
-          }
-        }
-
-        widget.div = $el('div', {})
-
-        document.body.appendChild(widget.div)
-
-        const inputDiv = (key, placeholder) => {
-          let div = document.createElement('div')
-          const ip = document.createElement('input')
-          ip.type = placeholder === 'Key' ? 'password' : 'text'
-          ip.className = `${'comfy-multiline-input'} ${placeholder}`
-          div.style = `display: flex;
-          align-items: center; 
-          margin: 6px 8px;
-          margin-top: 0;`
-          ip.placeholder = placeholder
-          ip.value = placeholder
-
-          ip.style = `margin-left: 24px;
-          outline: none;
-          border: none;
-          padding: 4px;width: 100%;`
-          const label = document.createElement('label')
-          label.style = 'font-size: 10px;min-width:32px'
-          label.innerText = placeholder
-          div.appendChild(label)
-          div.appendChild(ip)
-
-          ip.addEventListener('change', () => {
-            let data = getLocalData(key)
-            data[this.id] = ip.value.trim()
-            localStorage.setItem(key, JSON.stringify(data))
-            console.log(this.id, key)
-          })
-          return div
-        }
-
-        let inputKey = inputDiv('_mixlab_api_key', 'Key')
-        let inputUrl = inputDiv('_mixlab_api_url', 'URL')
-
-        widget.div.appendChild(inputKey)
-        widget.div.appendChild(inputUrl)
-
-        this.addCustomWidget(widget)
-
-        const onRemoved = this.onRemoved
-        this.onRemoved = () => {
-          inputUrl.remove()
-          inputKey.remove()
-          widget.div.remove()
-          return onRemoved?.()
-        }
-
-        this.serialize_widgets = true //需要保存参数
-      }
-    }
-  },
-  async loadedGraphNode (node, app) {
-    // Fires every time a node is constructed
-    // You can modify widgets/add handlers/etc here
-
-    if (node.type === 'ChatGPTOpenAI') {
-      let widget = node.widgets.filter(w => w.div)[0]
-
-      let apiKey = getLocalData('_mixlab_api_key'),
-        url = getLocalData('_mixlab_api_url')
-
-      let id = node.id
-
-      // console.log('ChatGPTOpenAI serialize_widgets', this)
-
-      widget.div.querySelector('.Key').value = apiKey[id] || 'by Mixlab'
-      widget.div.querySelector('.URL').value =
-        url[id] || 'https://api.openai.com/v1'
-    }
-  }
-})
 
 app.registerExtension({
   name: 'Mixlab.GPT.ShowTextForGPT',
@@ -209,13 +9,16 @@ app.registerExtension({
         text = text.filter(t => t && t?.trim())
 
         if (this.widgets) {
+          // console.log('#ShowTextForGPT',this.widgets)
           // const pos = this.widgets.findIndex(w => w.name === 'text')
           for (let i = 0; i < this.widgets.length; i++) {
-            if (this.widgets[i].name == 'show_text') this.widgets[i].onRemove?.()
+            if (this.widgets[i].name == 'show_text')
+              this.widgets[i].onRemove?.()
+       
           }
-          this.widgets.length = 1
+          this.widgets.length = 2
         }
-        // console.log('ShowTextForGPT',text)
+
         for (let list of text) {
           if (list) {
             // console.log('#####', list)
@@ -228,6 +31,8 @@ app.registerExtension({
             w.inputEl.readOnly = true
             w.inputEl.style.opacity = 0.6
 
+            // w.inputEl.style.display='none'
+
             try {
               if (typeof list != 'string') {
                 let data = JSON.parse(list)

web/javascript/image_mixlab.js

@@ -675,6 +675,17 @@ const createInputImageForBatch = (base64, widget) => {
   return im
 }
 
+// 添加新图片
+const addBase64ToWidgetForLoadImagesToBatch = (
+  base64,
+  imagesWidget,
+  imagesDiv
+) => {
+  imagesWidget.value.base64.push(base64)
+  let im = createInputImageForBatch(base64, imagesWidget)
+  imagesDiv.appendChild(im)
+}
+
 app.registerExtension({
   name: 'Mixlab.Comfy.LoadImagesToBatch',
   async getCustomWidgets (app) {
@@ -705,7 +716,6 @@ app.registerExtension({
 
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
     if (nodeType.comfyClass == 'LoadImagesToBatch') {
-
       const orig_nodeCreated = nodeType.prototype.onNodeCreated
 
       nodeType.prototype.onNodeCreated = function () {
@@ -751,13 +761,18 @@ app.registerExtension({
             base64 = await loadImageToCanvas(base64)
             // console.log(base64)
             if (!imagesWidget.value) imagesWidget.value = { base64: [] }
-            imagesWidget.value.base64.push(base64)
-            let im = createInputImageForBatch(base64, imagesWidget)
-            imagesDiv.appendChild(im)
+            addBase64ToWidgetForLoadImagesToBatch(
+              base64,
+              imagesWidget,
+              imagesDiv
+            )
           }
           reader.readAsDataURL(file)
         })
 
+        // 如果是复制的，有数据 , 这个不生效，取不到数据， 需要在nodeCreated里获取
+        // console.log('#LoadImagesToBatch', imagesWidget.value?.base64)
+
         const btn = document.createElement('button')
         btn.innerText = 'Upload Image'
 
@@ -829,18 +844,36 @@ app.registerExtension({
       }
     }
   },
+
   async loadedGraphNode (node, app) {
     if (node.type === 'LoadImagesToBatch') {
-      // await sleep(0)
       let imagesWidget = node.widgets.filter(w => w.name === 'images')[0]
       let imagePreview = node.widgets.filter(w => w.name == 'image_base64')[0]
+      // console.log('#LoadImagesToBatch', imagesWidget.value?.base64)
+      let imagesDiv = imagePreview.div.querySelector('.images_preview')
 
-      let pre = imagePreview.div.querySelector('.images_preview')
       for (const d of imagesWidget.value?.base64 || []) {
         let im = createInputImageForBatch(d, imagesWidget)
-        pre.appendChild(im)
+        imagesDiv.appendChild(im)
       }
     }
+  },
+  nodeCreated (node, app) {
+    //数据延迟？？
+    setTimeout(() => {
+      // console.log('#LoadImagesToBatch', node.type)
+      if (node.type === 'LoadImagesToBatch') {
+        let imagesWidget = node.widgets.filter(w => w.name === 'images')[0]
+        let imagePreview = node.widgets.filter(w => w.name == 'image_base64')[0]
+
+        let imagesDiv = imagePreview?.div?.querySelector('.images_preview')
+
+        for (const d of imagesWidget.value?.base64 || []) {
+          let im = createInputImageForBatch(d, imagesWidget)
+          imagesDiv.appendChild(im)
+        }
+      }
+    }, 1000)
   }
 })
 
@@ -868,8 +901,8 @@ app.registerExtension({
       const onNodeCreated = nodeType.prototype.onNodeCreated
       nodeType.prototype.onNodeCreated = function () {
         const r = onNodeCreated
-        ? onNodeCreated.apply(this, arguments)
-        : undefined
+          ? onNodeCreated.apply(this, arguments)
+          : undefined
 
         this.size = [400, this.size[1]]
         console.log('##onNodeCreated', this)
@@ -891,20 +924,15 @@ app.registerExtension({
         this.addCustomWidget(widget)
         this.serialize_widgets = true //需要保存参数
 
-
         const onRemoved = this.onRemoved
         this.onRemoved = () => {
           widget.div.remove()
           return onRemoved?.()
         }
 
-
-       return r
-
+        return r
       }
 
-     
-
       const onExecuted = nodeType.prototype.onExecuted
       nodeType.prototype.onExecuted = function (message) {
         onExecuted?.apply(this, arguments)
@@ -964,7 +992,7 @@ app.registerExtension({
             label: 'After'
           }
         ]
-        this.size=[this.size[0],300]
+        this.size = [this.size[0], 300]
       }
     }
   },
@@ -974,7 +1002,6 @@ app.registerExtension({
       // node.widgets[0].div.id = 'mix_comparingtowframes_' + node.id
       // if (node.widgets_values && node.widgets_values[0]) {
       //   node.widgets[0].div.innerHTML = ''
-
       //   let slider = new juxtapose.JXSlider(
       //     '#mix_comparingtowframes_' + node.id,
       //     node.widgets_values,

web/javascript/main_mixlab.js

@@ -1267,7 +1267,7 @@ app.registerExtension({
         })
 
         widget.PictureInPicture = $el('button', {
-          innerText: 'PictureInPicture',
+          innerText: 'Picture In Picture',
           style: {
             display: 'pictureInPictureEnabled' in document ? 'block' : 'none',
             cursor: 'pointer',

web/javascript/p5.js

@@ -0,0 +1,214 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { $el } from '../../../scripts/ui.js'
+
+function base64ToBlobFromURL (base64URL, contentType) {
+  return fetch(base64URL).then(response => response.blob())
+}
+
+async function uploadImage (blob, fileType = '.svg', filename) {
+  // const blob = await (await fetch(src)).blob();
+  const body = new FormData()
+  body.append(
+    'image',
+    new File([blob], (filename || new Date().getTime()) + fileType)
+  )
+
+  const resp = await api.fetchApi('/upload/image', {
+    method: 'POST',
+    body
+  })
+
+  // console.log(resp)
+  let data = await resp.json()
+  let { name, subfolder } = data
+  // let src = api.apiURL(
+  //   `/view?filename=${encodeURIComponent(
+  //     name
+  //   )}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+  // )
+
+  return data
+}
+// 上传得到url
+async function uploadBase64ToFile (base64) {
+  let bg_blob = await base64ToBlobFromURL(base64)
+  let url = await uploadImage(bg_blob, '.png')
+  return url
+}
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const p5InputNode = {
+  name: 'Mixlab.Comfy.P5Input',
+  async getCustomWidgets (app) {
+    return {
+      IMAGEBASE64 (node, inputName, inputData, app) {
+        const widget = {
+          value: {
+            images: []
+          }, // 不能[x,x,x]
+          type: inputData[0], // the type
+          name: inputName, // the name, slice
+          size: [320, 120], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          }
+        }
+        node.addCustomWidget(widget)
+        return widget
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'P5Input') {
+      console.log('P5Input')
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const widget = {
+          type: 'div',
+          name: 'image_base64',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width - 24, 44, node.size[1])
+            )
+          },
+          serialize: false
+        }
+
+        widget.div = $el('div', {})
+
+        widget.div.style = `margin:12px;width:400px;height:480px;background:white`
+
+        document.body.appendChild(widget.div)
+
+        this.addCustomWidget(widget)
+
+        // document.addEventListener('wheel', handleMouseWheel)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          // window.removeEventListener('message', ms)
+          return onRemoved?.()
+        }
+
+        // 节点的大小控制
+        this.setSize([480, 560])
+        app.canvas.draw(true, true)
+
+        const onResize = this.onResize
+        this.onResize = () => {
+          // 设置最小尺寸
+          if (
+            Math.max(this.size[0], 480) != this.size[0] &&
+            Math.max(this.size[1], 560) != this.size[1]
+          ) {
+            this.setSize([
+              Math.max(this.size[0], 480),
+              Math.max(this.size[1], 560)
+            ])
+          }
+
+          return onResize?.apply(this, arguments)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        // console.log('##onExecuted', this, message._info)
+        //   app.graph.getNodeById(8).widgets[1].div.querySelector('iframe').contentWindow.postMessage('Hello from parent', '*');
+      }
+    }
+  },
+
+  async loadedGraphNode (node, app) {
+    if (node.type === 'P5Input') {
+    }
+  },
+  nodeCreated (node, app) {
+    //数据延迟？？
+    setTimeout(() => {
+      let widget = node.widgets?.filter(w => w.name == 'image_base64')[0]
+      let framesWidget = node.widgets?.filter(w => w.name == 'frames')[0]
+      if (node.type === 'P5Input' && widget) {
+        if (framesWidget && !framesWidget.value)
+          framesWidget.value = { images: [] }
+
+        framesWidget.value._seed = Math.random()
+
+        let nodeId = node.id
+        //延迟才能获得this.id
+        widget.div.innerHTML = `<iframe src="extensions/comfyui-mixlab-nodes/p5_export/p5.html?id=${nodeId}" 
+          style="border:0;width:100%;height:100%;"
+         ></iframe>`
+
+        // 监听来自iframe的消息
+        const ms = async event => {
+          const data = event.data
+          if (
+            data.from === 'p5.widget' &&
+            data.status === 'save' &&
+            data.frames &&
+            data.frames.length > 0 &&
+            data.nodeId == nodeId
+          ) {
+            const frames = data.frames
+            console.log(frames.length, nodeId)
+            //workflow会存储到local，会卡死
+            framesWidget.value.images = []
+            for (const f of frames) {
+              let file = await uploadBase64ToFile(f)
+              framesWidget.value.images.push(file)
+            }
+            // framesWidget.value.base64 = frames
+            framesWidget.value._seed = Math.random()
+            node.title = 'P5 Input #' + frames.length
+          }
+        }
+        window.addEventListener('message', ms)
+      }
+    }, 1000)
+  }
+}
+
+app.registerExtension(p5InputNode)

web/javascript/sound-lab.js

@@ -0,0 +1,295 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+import WaveSurfer from 'https://cdn.jsdelivr.net/npm/wavesurfer.js@7/dist/wavesurfer.esm.js'
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: '0',
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+//把文件转为url访问
+const parseUrl = data => {
+  let { filename, subfolder, type, prompt } = data
+  return {
+    url: api.apiURL(
+      `/view?filename=${encodeURIComponent(
+        filename
+      )}&type=${type}&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+    ),
+    prompt
+  }
+}
+
+const createWaveSurfer = (wavesurfer, id,url) => {
+  // Create an instance of WaveSurfer
+  if (wavesurfer) {
+    wavesurfer.destroy()
+  }
+  wavesurfer = WaveSurfer.create({
+    container: '#' + id,
+    waveColor: 'rgb(200, 0, 200)',
+    progressColor: 'rgb(100, 0, 100)',
+    // Set a bar width
+    barWidth: 10,
+    // Optionally, specify the spacing between bars
+    barGap: 2,
+    // And the bar radius
+    barRadius: 6,
+    url
+  })
+
+  wavesurfer._auto = true
+
+  // 监听播放结束事件，重新开始播放以实现循环播放
+  wavesurfer.on('finish', function () {
+    // console.log(wavesurfer)
+    if (wavesurfer._auto) wavesurfer.play()
+  })
+
+  wavesurfer.on('interaction', () => {
+    wavesurfer._auto = false
+    if (!wavesurfer.isPlaying()) wavesurfer.play()
+  })
+
+  // 获取当前播放时间的峰值
+  wavesurfer.on('audioprocess', () => {
+    if (wavesurfer.isPlaying()&&wavesurfer.getDecodedData()) {
+      const channelData = wavesurfer.getDecodedData().getChannelData(0); 
+      const currentTime = wavesurfer.getCurrentTime()
+      // console.log(wavesurfer)
+      const sampleRate = wavesurfer.getDecodedData().sampleRate
+
+      // 定义要分析的时间窗口（例如1秒）
+      const windowSize = 1
+      const startSample = Math.floor(currentTime * sampleRate)
+      const endSample = Math.min(
+        startSample + windowSize * sampleRate,
+        channelData.length
+      )
+
+      let peak = 0
+      for (let i = startSample; i < endSample; i++) {
+        const value = Math.abs(channelData[i])
+        if (value > peak) {
+          peak = value
+        }
+      }
+      // console.log('Current Peak:', peak)
+    }
+  })
+
+  return wavesurfer
+}
+
+//更新gui
+function updateWaveWidgetValue (widgets, id, url, prompt, wavesurfer) {
+  let widget = widgets.filter(w => w.name == 'AudioPlay')[0]
+  // 手动更新widget值
+  widget.value = [url, prompt]
+
+  if (widget.div) {
+    widget.div.querySelector('.wave').id = `AudioPlay_${id}`
+  }
+
+  wavesurfer = createWaveSurfer(wavesurfer, `AudioPlay_${id}`,url)
+
+  wavesurfer.on('ready', duration => {
+    console.log('Audio duration: ' + duration + ' seconds')
+    if (widget.div) {
+      widget.div.setAttribute('data-url', url)
+      widget.div.querySelector('.link').setAttribute('href', url)
+      widget.div.querySelector(
+        '.info'
+      ).innerHTML = `<span style="font-size: 12px;
+      margin: 8px;">${duration.toFixed(
+        2
+      )} seconds</span> <br><span style="font-size: 14px;">${prompt||''}</span> <br>`
+    }
+  })
+  
+  
+  wavesurfer.load(url)
+  // console.log('updateWaveWidgetValue' ,url,wavesurfer)
+  return wavesurfer
+}
+
+app.registerExtension({
+  name: 'SoundLab.AudioPlay',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'AudioPlay') {
+      let that = this
+      // console.log('that', that)
+
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const widget = {
+          type: 'div',
+          name: 'AudioPlay',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, y, node.size[1])
+            )
+          }
+        }
+
+        // console.log('AudioPlay nodeData', this)
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        // wave
+        const waveDiv = document.createElement('div')
+        waveDiv.className = 'wave'
+        waveDiv.style.minHeight = '172px'
+        widget.div.appendChild(waveDiv)
+
+        //prompt 相关信息展示
+        const infoDiv = document.createElement('div')
+        infoDiv.className = 'info'
+        infoDiv.style.marginBottom = '20px'
+        widget.div.appendChild(infoDiv)
+
+        // 按钮的区域
+        let btns = document.createElement('div')
+        btns.className = 'btns'
+        btns.style = `display: flex;
+        width: 100%;
+        justify-content: space-between;`
+        widget.div.appendChild(btns)
+
+        //play button
+        const playBtn = document.createElement('a')
+        playBtn.innerText = 'Play/Pause'
+
+        playBtn.style = `
+        display: flex;
+        padding: 4px 15px;
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;
+        color: var(--descrip-text);
+        text-decoration: none;
+        border-radius: 5px;
+        transition: background-color 0.3s ease 0s;
+          `
+
+        playBtn.addEventListener('click', e => {
+          e.preventDefault()
+          if (that[`wavesurfer_${this.id}`]) {
+            that[`wavesurfer_${this.id}`]?.playPause()
+            that[`wavesurfer_${this.id}`]._auto = true
+          }
+        })
+        btns.appendChild(playBtn)
+
+        const urlLink = document.createElement('a')
+        urlLink.className = 'link'
+        urlLink.innerText = 'URL'
+        urlLink.setAttribute('target', '_blank')
+        urlLink.style = `display: flex;
+        padding: 4px 15px;
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;
+        color: var(--descrip-text);
+        text-decoration: none;
+        border-radius: 5px;
+        transition: background-color 0.3s ease 0s;`
+        // urlLink.style.minHeight = '200px'
+        btns.appendChild(urlLink)
+
+
+        //todo 导出视频 that[`wavesurfer_${this.id}`].renderer.exportImage('image/png',1,'dataURL')
+        // https://github.com/diffusion-studio/ffmpeg-js
+
+
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        this.size = [this.size[0], 280]
+        this.serialize_widgets = true //需保存widget的值
+      }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        const audio = message.audio
+        console.log('#onExecuted', `AudioPlay_${this.id}`, message,audio)
+        try {
+          let { url, prompt } = parseUrl(audio[0])
+
+          that[`wavesurfer_${this.id}`] = updateWaveWidgetValue(
+            this.widgets,
+            this.id,
+            url,
+            prompt,
+            that[`wavesurfer_${this.id}`]
+          )
+
+          that[`wavesurfer_${this.id}`]?.playPause()
+        } catch (error) {
+          console.log(error)
+        }
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'AudioPlay') {
+      let widget = node.widgets.filter(w => w.name == 'AudioPlay')[0]
+
+      if (widget.value) {
+        let [url, prompt] = widget.value
+
+        this[`wavesurfer_${node.id}`] = updateWaveWidgetValue(
+          node.widgets,
+          node.id,
+          url,
+          prompt,
+          this[`wavesurfer_${node.id}`]
+        )
+      }
+
+      console.log('#loadedGraphNode', node)
+    }
+  }
+})

web/javascript/td_background.js

@@ -0,0 +1,323 @@
+// touchdesigner的背景效果，把appinfo的输出，选择一张图片作为背景
+
+window._bg_img = null
+
+/**
+ * draws the back canvas (the one containing the background and the connections)
+ * @method drawBackCanvas
+ **/
+LGraphCanvas.prototype.drawBackCanvas = function () {
+  var canvas = this.bgcanvas
+  if (
+    canvas.width != this.canvas.width ||
+    canvas.height != this.canvas.height
+  ) {
+    canvas.width = this.canvas.width
+    canvas.height = this.canvas.height
+  }
+
+  if (!this.bgctx) {
+    this.bgctx = this.bgcanvas.getContext('2d')
+  }
+  var ctx = this.bgctx
+  if (ctx.start) {
+    ctx.start()
+  }
+
+  var viewport = this.viewport || [0, 0, ctx.canvas.width, ctx.canvas.height]
+
+  //clear
+  if (this.clear_background) {
+    ctx.clearRect(viewport[0], viewport[1], viewport[2], viewport[3])
+  }
+
+  //show subgraph stack header
+  if (this._graph_stack && this._graph_stack.length) {
+    ctx.save()
+    var parent_graph = this._graph_stack[this._graph_stack.length - 1]
+    var subgraph_node = this.graph._subgraph_node
+    ctx.strokeStyle = subgraph_node.bgcolor
+    ctx.lineWidth = 10
+    ctx.strokeRect(1, 1, canvas.width - 2, canvas.height - 2)
+    ctx.lineWidth = 1
+    ctx.font = '40px Arial'
+    ctx.textAlign = 'center'
+    ctx.fillStyle = subgraph_node.bgcolor || '#AAA'
+    var title = ''
+    for (var i = 1; i < this._graph_stack.length; ++i) {
+      title += this._graph_stack[i]._subgraph_node.getTitle() + ' >> '
+    }
+    ctx.fillText(title + subgraph_node.getTitle(), canvas.width * 0.5, 40)
+    ctx.restore()
+  }
+
+  var bg_already_painted = false
+  if (this.onRenderBackground) {
+    bg_already_painted = this.onRenderBackground(canvas, ctx)
+  }
+
+  //reset in case of error
+  if (!this.viewport) {
+    ctx.restore()
+    ctx.setTransform(1, 0, 0, 1, 0, 0)
+  }
+  this.visible_links.length = 0
+
+  if (this.graph) {
+    //apply transformations
+    ctx.save()
+    this.ds.toCanvasContext(ctx)
+
+    //render BG
+    if (
+      this.ds.scale < 1 &&
+      !bg_already_painted &&
+      this.clear_background_color
+    ) {
+      ctx.fillStyle = this.clear_background_color
+      ctx.fillRect(
+        this.visible_area[0],
+        this.visible_area[1],
+        this.visible_area[2],
+        this.visible_area[3]
+      )
+    }
+
+    // 主要修改
+    if (this.background_image && this.ds.scale > 0.5 && !bg_already_painted) {
+      if (this.zoom_modify_alpha) {
+        //使得 alpha 越接近0时变化越缓慢。
+        let alpha = (1.0 - 0.5 / this.ds.scale) * this.editor_alpha
+        ctx.globalAlpha = Math.min(Math.max(0, Math.sqrt(alpha)), 1)
+        // console.log((1.0 - 0.5 / this.ds.scale) * this.editor_alpha)
+      } else {
+        ctx.globalAlpha = this.editor_alpha
+      }
+      ctx.imageSmoothingEnabled = ctx.imageSmoothingEnabled = false // ctx.mozImageSmoothingEnabled =
+      if (!this._bg_img || this._bg_img.name != this.background_image) {
+        this._bg_img = new Image()
+        this._bg_img.name = this.background_image
+        this._bg_img.src = this.background_image
+        var that = this
+        this._bg_img.onload = function () {
+          that.draw(true, true)
+        }
+      }
+
+      var pattern = null
+      if (this._pattern == null && this._bg_img.width > 0) {
+        pattern = ctx.createPattern(this._bg_img, 'repeat')
+        this._pattern_img = this._bg_img
+        this._pattern = pattern
+      } else {
+        pattern = this._pattern
+      }
+
+      if (pattern) {
+        ctx.fillStyle = pattern
+        ctx.fillRect(
+          this.visible_area[0],
+          this.visible_area[1],
+          this.visible_area[2],
+          this.visible_area[3]
+        )
+        ctx.fillStyle = 'transparent'
+      }
+
+      ctx.globalAlpha = 1.0
+      ctx.imageSmoothingEnabled = ctx.imageSmoothingEnabled = true //= ctx.mozImageSmoothingEnabled
+    }
+
+    //groups
+    if (this.graph._groups.length && !this.live_mode) {
+      this.drawGroups(canvas, ctx)
+    }
+
+    if (this.onDrawBackground) {
+      this.onDrawBackground(ctx, this.visible_area)
+    }
+    if (this.onBackgroundRender) {
+      //LEGACY
+      console.error(
+        'WARNING! onBackgroundRender deprecated, now is named onDrawBackground '
+      )
+      this.onBackgroundRender = null
+    }
+
+    //DEBUG: show clipping area
+    //ctx.fillStyle = "red";
+    //ctx.fillRect( this.visible_area[0] + 10, this.visible_area[1] + 10, this.visible_area[2] - 20, this.visible_area[3] - 20);
+
+    //bg
+    if (this.render_canvas_border) {
+      ctx.strokeStyle = '#235'
+      ctx.strokeRect(0, 0, canvas.width, canvas.height)
+    }
+
+    if (this.render_connections_shadows) {
+      ctx.shadowColor = '#000'
+      ctx.shadowOffsetX = 0
+      ctx.shadowOffsetY = 0
+      ctx.shadowBlur = 6
+    } else {
+      ctx.shadowColor = 'rgba(0,0,0,0)'
+    }
+
+    //draw connections
+    if (!this.live_mode) {
+      this.drawConnections(ctx)
+    }
+
+    ctx.shadowColor = 'rgba(0,0,0,0)'
+
+    //restore state
+    ctx.restore()
+  }
+
+  if (ctx.finish) {
+    ctx.finish()
+  }
+
+  this.dirty_bgcanvas = false
+  this.dirty_canvas = true //to force to repaint the front canvas with the bgcanvas
+}
+
+function imgToCanvasBase64 (img) {
+  const canvas = document.createElement('canvas')
+  const ctx = canvas.getContext('2d')
+  canvas.width = img.width
+  canvas.height = img.height
+  ctx.drawImage(img, 0, 0)
+  const base64 = canvas.toDataURL('image/png')
+
+  return base64
+}
+
+// 使用示例
+function convertImageToBase64 (img) {
+  //   const img = new Image()
+  //   img.src = 'path/to/your/image.jpg' // 替换为你的图片路径
+  // console.log('convertImageToBase64',img)
+  try {
+    const base64 = imgToCanvasBase64(img)
+    return base64
+  } catch (error) {
+    console.error(error)
+  }
+}
+
+function getInputsAndOutputs () {
+  const outputs =
+    `PreviewImage,SaveImage,TransparentImage,VHS_VideoCombine,VideoCombine_Adv,Image Save,SaveImageAndMetadata_`.split(
+      ','
+    )
+
+  let outputsId = []
+
+  for (let node of app.graph._nodes) {
+    if (outputs.includes(node.type)) {
+      outputsId.push(node.id)
+    }
+  }
+
+  return outputsId
+}
+
+function getRandomElement (arr) {
+  const randomIndex = Math.floor(Math.random() * arr.length)
+  return arr[randomIndex]
+}
+
+async function getBG () {
+  var outputs = []
+
+  for (let id of app.graph
+    .getNodeById(50)
+    .widgets.filter(w => w.name === 'output_ids')[0]
+    .value.split('\n')) {
+    if (getInputsAndOutputs().map(Number).includes(Number(id))) {
+      if (app.graph.getNodeById(id).imgs && app.graph.getNodeById(id).imgs[0]) {
+        let b = convertImageToBase64(app.graph.getNodeById(id).imgs[0])
+        // console.log(b)
+        outputs.push(b)
+      }
+    }
+  }
+
+  var BACKGROUND_IMAGE = getRandomElement(outputs),
+    CLEAR_BACKGROUND_COLOR = 'rgba(0,0,0,0.9)'
+
+  if (!window._bg_img) {
+    window._bg_img = app.canvas._bg_img.src
+  }
+  // let img=new Image();
+  // img.src=BACKGROUND_IMAGE;
+
+  //去掉透明度过度
+  // app.canvas.zoom_modify_alpha=false;
+  //整体透明度
+  app.canvas.editor_alpha = 1.1
+  //   app.canvas._pattern=ctx.createPattern(img, "no-repeat");
+  app.canvas.updateBackground(BACKGROUND_IMAGE, CLEAR_BACKGROUND_COLOR)
+  app.canvas.draw(true, true)
+}
+
+class BgRunner {
+  constructor () {
+    this.intervalId = null
+    this.running = false
+  }
+
+  // 要运行的方法
+  bg () {
+    console.log('方法bg正在运行')
+    getBG()
+  }
+
+  // 启动bg方法每秒运行一次
+  start () {
+    if (!this.running) {
+      this.intervalId = setInterval(() => this.bg(), 1500)
+      this.running = true
+    }
+  }
+
+  // 停止bg方法的运行
+  stop () {
+    if (this.running) {
+      clearInterval(this.intervalId)
+      this.intervalId = null
+      this.running = false
+
+      if (window._bg_img) {
+        var BACKGROUND_IMAGE = window._bg_img,
+          CLEAR_BACKGROUND_COLOR = 'rgba(0,0,0,1)'
+        app.canvas.editor_alpha = 1
+
+        app.canvas.updateBackground(BACKGROUND_IMAGE, CLEAR_BACKGROUND_COLOR)
+        app.canvas.draw(true, true)
+      }
+    }
+  }
+
+  // 切换start和stop
+  toggle () {
+    if (this.running) {
+      this.stop()
+    } else {
+      this.start()
+    }
+  }
+
+  // 获取运行状态
+  isRunning () {
+    return this.running
+  }
+}
+
+// 示例用法
+// const runner = new BgRunner();
+// runner.start();
+// setTimeout(() => runner.stop(), 5000);
+
+export const td_bg = new BgRunner()

web/javascript/ui_mixlab.js

@@ -100,7 +100,7 @@ async function start_llama (model = 'Phi-3-mini-4k-instruct-Q5_K_S.gguf') {
     })
 
     const data = await response.json()
-    if (data.llama_cpp_error) {
+    if (data.llama_cpp_error||!data.port) {
       return
     }
 
@@ -163,17 +163,20 @@ async function createMenu () {
 
     // appsButton.onclick = () =>
     appsButton.onclick = async () => {
-      if (window._mixlab_llamacpp) {
-        //显示运行的模型
-        createModelsModal([
-          window._mixlab_llamacpp.url,
-          window._mixlab_llamacpp.model
-        ])
-      } else {
-        let ms = await get_llamafile_models()
-        ms = ms.filter(m => !m.match('-mmproj-'))
-        if (ms.length > 0) createModelsModal(ms)
-      }
+      // if (window._mixlab_llamacpp&&window._mixlab_llamacpp.model&&window._mixlab_llamacpp.model.length>0) {
+      //   //显示运行的模型
+      //   createModelsModal([
+      //     window._mixlab_llamacpp.url,
+      //     window._mixlab_llamacpp.model
+      //   ])
+      // } else {
+      //   // let ms = await get_llamafile_models()
+      //   // ms = ms.filter(m => !m.match('-mmproj-'))
+      //   // if (ms.length > 0) createModelsModal(ms)
+      // }
+      createModelsModal([
+      
+      ])
     }
     menu.append(appsButton)
   }
@@ -800,11 +803,11 @@ async function fetchReadmeContent (url) {
 
 async function startLLM (model) {
   let res = await start_llama(model)
-  window._mixlab_llamacpp = res
+  window._mixlab_llamacpp = res||{ model:[] }
 
-  localStorage.setItem('_mixlab_llama_select', res.model)
+  localStorage.setItem('_mixlab_llama_select', res?.model||'')
 
-  if (document.body.querySelector('#mixlab_chatbot_by_llamacpp')&&window._mixlab_llamacpp.url) {
+  if (document.body.querySelector('#mixlab_chatbot_by_llamacpp')&&window._mixlab_llamacpp?.url) {
     document.body
       .querySelector('#mixlab_chatbot_by_llamacpp')
       .setAttribute('title', window._mixlab_llamacpp.url)
@@ -932,16 +935,16 @@ function createModelsModal (models) {
   const n_gpu_p = document.createElement('p')
   n_gpu_p.innerText = 'n_gpu_layers'
 
-  const n_gpu_div = document.createElement('div')
-  n_gpu_div.style = `display: flex;
+  const batchPageBtn = document.createElement('div')
+  batchPageBtn.style = `display: flex;
   justify-content: center;
   align-items: center;
   font-size: 12px;`
-  n_gpu_div.appendChild(n_gpu_p)
-  n_gpu_div.appendChild(n_gpu)
+  batchPageBtn.innerHTML=`<a href="${get_url()}/mixlab/app" target="_blank" style="color: var(--input-text);
+  background-color: var(--comfy-input-bg);">App</a>`
 
   const title = document.createElement('p')
-  title.innerText = 'Models'
+  title.innerText = 'Mixlab Nodes'
   title.style = `font-size: 18px;
   margin-right: 8px;
   margin-top: 0;`
@@ -953,9 +956,9 @@ function createModelsModal (models) {
   font-size: 12px;
   flex-direction: column; `
   left_d.appendChild(title)
-  title.appendChild(statusIcon)
-  left_d.appendChild(linkIcon)
-  left_d.appendChild(n_gpu_div)
+  // title.appendChild(statusIcon)
+  // left_d.appendChild(linkIcon)
+  left_d.appendChild(batchPageBtn)
   headTitleElement.appendChild(left_d)
 
   // headTitleElement.appendChild(n_gpu_div)
@@ -1010,26 +1013,26 @@ function createModelsModal (models) {
   var modalContent = document.createElement('div')
   modalContent.classList.add('modal-content')
 
-  var input = document.createElement('textarea')
-  input.className = 'comfy-multiline-input'
-  input.style = `    height: 260px;
+  var inputForSystemPrompt = document.createElement('textarea')
+  inputForSystemPrompt.className = 'comfy-multiline-input'
+  inputForSystemPrompt.style = `    height: 260px;
   width: 480px;
   font-size: 16px;
   padding: 18px;`
-  input.value = localStorage.getItem('_mixlab_system_prompt')
+  inputForSystemPrompt.value = localStorage.getItem('_mixlab_system_prompt')
 
-  input.addEventListener('change', e => {
+  inputForSystemPrompt.addEventListener('change', e => {
     e.stopPropagation()
-    localStorage.setItem('_mixlab_system_prompt', input.value)
+    localStorage.setItem('_mixlab_system_prompt', inputForSystemPrompt.value)
   })
 
-  input.addEventListener('click', e => {
+  inputForSystemPrompt.addEventListener('click', e => {
     e.stopPropagation()
   })
 
-  modalContent.appendChild(input)
+  // modalContent.appendChild(inputForSystemPrompt)
 
-  if (!window._mixlab_llamacpp) {
+  if (!window._mixlab_llamacpp||(window._mixlab_llamacpp?.model?.length==0)) {
     for (const m of models) {
       let d = document.createElement('div')
       d.innerText = `${showTextByLanguage('Run', {
@@ -1040,10 +1043,10 @@ function createModelsModal (models) {
       d.addEventListener('click', async e => {
         e.stopPropagation()
         div.remove()
-        startLLM(m)
+        // startLLM(m)
       })
 
-      modalContent.appendChild(d)
+      // modalContent.appendChild(d)
     }
   }
   modal.appendChild(modalContent)
@@ -1414,7 +1417,7 @@ app.registerExtension({
           .setAttribute('title', res.url)
       })
     }else{
-      startLLM('')
+      // startLLM('')
     }
 
     LGraphCanvas.prototype.helpAboutNode = async function (node) {
@@ -1439,10 +1442,14 @@ app.registerExtension({
 
     LGraphCanvas.prototype.fixTheNode = function (node) {
       let new_node = LiteGraph.createNode(node.comfyClass)
-      new_node.pos = [node.pos[0], node.pos[1]]
-      app.canvas.graph.add(new_node, false)
-      copyNodeValues(node, new_node)
-      app.canvas.graph.remove(node)
+      console.log(node)
+      if(new_node){
+        new_node.pos = [node.pos[0], node.pos[1]]
+        app.canvas.graph.add(new_node, false)
+        copyNodeValues(node, new_node)
+        app.canvas.graph.remove(node)
+      }
+     
     }
 
     smart_init()
@@ -1784,6 +1791,7 @@ app.registerExtension({
         {
           content: 'Help ♾️Mixlab', // with a name
           callback: () => {
+            // console.log('#data',node)
             LGraphCanvas.prototype.helpAboutNode(node)
           } // and the callback
         },

web/javascript/utils_mixlab.js

@@ -1,5 +1,5 @@
 import { app } from '../../../scripts/app.js'
-import { $el } from '../../../scripts/ui.js' 
+import { $el } from '../../../scripts/ui.js'
 
 const getLocalData = key => {
   let data = {}
@@ -122,7 +122,7 @@ app.registerExtension({
       nodeType.prototype.onNodeCreated = function () {
         orig_nodeCreated?.apply(this, arguments)
 
-        // console.log('Color nodeData', this.widgets)
+        console.log('Color nodeData', this.div)
 
         const widget = {
           type: 'div',
@@ -273,19 +273,19 @@ app.registerExtension({
 })
 
 const min_max = node => {
-  if(node.widgets){
+  if (node.widgets) {
     const min_value = node.widgets.filter(w => w.name === 'min_value')[0]
     const max_value = node.widgets.filter(w => w.name === 'max_value')[0]
-  
+
     const number = node.widgets.filter(w => w.name === 'number')[0]
     if (number) {
       number.options.min = min_value.value
       number.options.max = max_value.value
-  
+
       number.value = Math.min(number.options.max, number.value)
       number.value = Math.max(number.options.min, number.value)
     }
-  
+
     if (min_value)
       min_value.callback = e => {
         number.options.min = e
@@ -297,22 +297,18 @@ const min_max = node => {
         number.value = e
       }
   }
-  
 }
 
 app.registerExtension({
   name: 'Mixlab.utils.FloatSlider',
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
-
     if (nodeType.comfyClass == 'FloatSlider') {
-      const orig_nodeCreated = nodeType.prototype.onNodeCreated;
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
       nodeType.prototype.onNodeCreated = function () {
         orig_nodeCreated?.apply(this, arguments)
         min_max(this)
       }
     }
-
-   
   },
   async loadedGraphNode (node, app) {
     if (node.type === 'FloatSlider') {
@@ -323,7 +319,6 @@ app.registerExtension({
 app.registerExtension({
   name: 'Mixlab.utils.IntNumber',
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
-    
     if (nodeType.comfyClass == 'IntNumber') {
       const orig_nodeCreated = nodeType.prototype.onNodeCreated
       nodeType.prototype.onNodeCreated = function () {
@@ -331,7 +326,6 @@ app.registerExtension({
         min_max(this)
       }
     }
-   
   },
   async loadedGraphNode (node, app) {
     if (node.type === 'IntNumber') {
@@ -340,22 +334,145 @@ app.registerExtension({
   }
 })
 
-
 app.registerExtension({
   name: 'Mixlab.utils.TESTNODE_',
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
-    
     if (nodeType.comfyClass == 'TESTNODE_') {
-    
-      const onExecuted = nodeType.prototype.onExecuted;
-			nodeType.prototype.onExecuted = function (message) {
-				onExecuted?.apply(this, arguments);
-        console.log('##',message)
-				 
-			};
-
-
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        console.log('##', message)
+      }
     }
-   
-  }, 
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.utils.KeyInput',
+  init () {},
+  async getCustomWidgets (app) {
+    return {
+      KEY (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 24], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let data = getLocalData('_mixlab_api_key')
+            return data[node.id] || 'by Mixlab'
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'KeyInput') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const widget = {
+          type: 'div',
+          name: 'input_key',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 24, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder) => {
+          let div = document.createElement('div')
+          div.style = `
+          display: flex;
+          align-items: center; 
+          margin: 6px 8px;
+          margin-top:0px;
+          height:44px;
+          width:220px; 
+          `
+
+          const ip = document.createElement('input')
+          ip.type = 'password'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+
+          ip.placeholder = placeholder
+          // ip.value = placeholder
+
+          ip.style = `margin-left:8px;
+          outline: none;
+          border: none;
+          padding:12px;
+          width: 100%;
+          `
+
+          div.appendChild(ip)
+
+          ip.addEventListener('change', () => {
+            let data = getLocalData(key)
+            data[this.id] = ip.value.trim()
+            localStorage.setItem(key, JSON.stringify(data))
+          })
+
+          return div
+        }
+
+        let inputKey = inputDiv('_mixlab_api_key', 'Key')
+
+        widget.div.appendChild(inputKey)
+
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          inputKey.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'KeyInput') {
+      let widget = node.widgets.filter(w => w.div)[0]
+
+      let apiKey = getLocalData('_mixlab_api_key')
+
+      let id = node.id
+      if (widget.div.querySelector('.Key'))
+        widget.div.querySelector('.Key').value = apiKey[id] || 'by Mixlab'
+    }
+  },
+  nodeCreated (node, app) {
+    //数据延迟？？
+    setTimeout(() => {
+      // console.log('#LoadImagesToBatch', node.type)
+      if (node.type === 'KeyInput') {
+        let widget = node.widgets.filter(w => w.div)[0]
+
+        let apiKey = getLocalData('_mixlab_api_key')
+
+        let id = node.id
+
+        if (widget.div.querySelector('.Key'))
+          widget.div.querySelector('.Key').value = apiKey[id] || 'by Mixlab'
+      }
+    }, 1000)
+  }
 })

web/javascript/video_mixlab.js

@@ -6,8 +6,6 @@ import { $el } from '../../../scripts/ui.js'
 
 // The code is based on ComfyUI-VideoHelperSuite modification.
 
-
-
 function injectCSS (css) {
   // 检查页面中是否已经存在具有相同内容的style标签
   const existingStyle = document.querySelector('style')
@@ -240,15 +238,7 @@ app.registerExtension({
   }
 })
 
-
-function offsetDOMWidget(
-  widget,
-  ctx,
-  node,
-  widgetWidth,
-  widgetY,
-  height
-) {
+function offsetDOMWidget (widget, ctx, node, widgetWidth, widgetY, height) {
   const margin = 10
   const elRect = ctx.canvas.getBoundingClientRect()
   const transform = new DOMMatrix()
@@ -270,18 +260,18 @@ function offsetDOMWidget(
     position: 'absolute',
     background: !node.color ? '' : node.color,
     color: !node.color ? '' : 'white',
-    zIndex: 5, //app.graph._nodes.indexOf(node),
+    zIndex: 5 //app.graph._nodes.indexOf(node),
   })
 }
 
-export const hasWidgets = (node) => {
+export const hasWidgets = node => {
   if (!node.widgets || !node.widgets?.[Symbol.iterator]) {
     return false
   }
   return true
 }
 
-export const cleanupNode = (node) => {
+export const cleanupNode = node => {
   if (!hasWidgets(node)) {
     return
   }
@@ -298,43 +288,43 @@ export const cleanupNode = (node) => {
   }
 }
 
-const CreatePreviewElement = (name, val, format) => {
-  const [type] = format.split('/')  
+const createPreviewElement = (name, val, format) => {
+  const [type] = format.split('/')
   const w = {
-      name,
-      type,
-      value: val,
-      draw: function (ctx, node, widgetWidth, widgetY, height) {
-        const [cw, ch] = this.computeSize(widgetWidth)
-        offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, ch)
-      },
-      computeSize: function (_) {
-        const ratio = this.inputRatio || 1
-        const width = Math.max(220, this.parent.size[0])
-        return [width, (width / ratio + 10)]
-      },
-      onRemoved: function () {
-        if (this.inputEl) {
-          this.inputEl.remove()
-        }
-      },
+    name,
+    type,
+    value: val,
+    draw: function (ctx, node, widgetWidth, widgetY, height) {
+      const [cw, ch] = this.computeSize(widgetWidth)
+      offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, ch)
+    },
+    computeSize: function (_) {
+      const ratio = this.inputRatio || 1
+      const width = Math.max(220, this.parent.size[0])
+      return [width, width / ratio + 10]
+    },
+    onRemoved: function () {
+      if (this.inputEl) {
+        this.inputEl.remove()
+      }
     }
-
-    w.inputEl = document.createElement(type === 'video' ? 'video' : 'img')
-    w.inputEl.src = w.value
-    if (type === 'video') {
-      w.inputEl.setAttribute('type', 'video/webm');
-      w.inputEl.autoplay = true
-      w.inputEl.loop = true
-      w.inputEl.controls = false;
-    }
-    w.inputEl.onload = function () {
-      w.inputRatio = w.inputEl.naturalWidth / w.inputEl.naturalHeight
-    }
-    document.body.appendChild(w.inputEl)
-    return w
   }
 
+  w.inputEl = document.createElement(type === 'video' ? 'video' : 'img')
+  w.inputEl.src = w.value
+
+  if (type === 'video' || format.match('.mp4')) {
+    w.inputEl.setAttribute('type', 'video/webm')
+    w.inputEl.autoplay = true
+    w.inputEl.loop = true
+    w.inputEl.controls = true
+  }
+  w.inputEl.onload = function () {
+    w.inputRatio = w.inputEl.naturalWidth / w.inputEl.naturalHeight
+  }
+  document.body.appendChild(w.inputEl)
+  return w
+}
 
 app.registerExtension({
   name: 'Mixlab.Video.ImageListReplace',
@@ -469,12 +459,17 @@ app.registerExtension({
       }
     }
 
-    if (nodeData?.name == 'VideoCombine_Adv') {
+    if (
+      nodeData?.name == 'VideoCombine_Adv' ||
+      nodeData?.name == 'CombineAudioVideo'
+    ) {
       const onExecuted = nodeType.prototype.onExecuted
       nodeType.prototype.onExecuted = function (message) {
         const prefix = 'vhs_gif_preview_'
         const r = onExecuted ? onExecuted.apply(this, message) : undefined
 
+        if(!this.widgets) this.widgets=[]
+
         if (this.widgets) {
           const pos = this.widgets.findIndex(w => w.name === `${prefix}_0`)
           if (pos !== -1) {
@@ -489,12 +484,13 @@ app.registerExtension({
                 '/view?' + new URLSearchParams(params).toString()
               )
               const w = this.addCustomWidget(
-                CreatePreviewElement(
+                createPreviewElement(
                   `${prefix}_${i}`,
                   previewUrl,
                   params.format || 'image/gif'
                 )
               )
+              console.log(w)
               w.parent = this
             })
           }

web/p5_export/p5-widget/p5-widget.html

@@ -0,0 +1,6 @@
+<!DOCTYPE html>
+<meta charset="utf-8">
+<link href="https://fonts.googleapis.com/css?family=Montserrat" rel="stylesheet">
+<title>p5.js-widget</title>
+<div id="app-holder"></div>
+<script src="./main.bundle.js"></script>

web/p5_export/p5-widget/p5-widget.js

@@ -0,0 +1,240 @@
+/******/ (function(modules) { // webpackBootstrap
+/******/ 	// The module cache
+/******/ 	var installedModules = {};
+/******/
+/******/ 	// The require function
+/******/ 	function __webpack_require__(moduleId) {
+/******/
+/******/ 		// Check if module is in cache
+/******/ 		if(installedModules[moduleId])
+/******/ 			return installedModules[moduleId].exports;
+/******/
+/******/ 		// Create a new module (and put it into the cache)
+/******/ 		var module = installedModules[moduleId] = {
+/******/ 			exports: {},
+/******/ 			id: moduleId,
+/******/ 			loaded: false
+/******/ 		};
+/******/
+/******/ 		// Execute the module function
+/******/ 		modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
+/******/
+/******/ 		// Flag the module as loaded
+/******/ 		module.loaded = true;
+/******/
+/******/ 		// Return the exports of the module
+/******/ 		return module.exports;
+/******/ 	}
+/******/
+/******/
+/******/ 	// expose the modules object (__webpack_modules__)
+/******/ 	__webpack_require__.m = modules;
+/******/
+/******/ 	// expose the module cache
+/******/ 	__webpack_require__.c = installedModules;
+/******/
+/******/ 	// __webpack_public_path__
+/******/ 	__webpack_require__.p = "";
+/******/
+/******/ 	// Load entry module and return exports
+/******/ 	return __webpack_require__(0);
+/******/ })
+/************************************************************************/
+/******/ ([
+/* 0 */
+/***/ (function(module, exports, __webpack_require__) {
+
+	"use strict";
+	var defaults = __webpack_require__(1);
+	var MY_FILENAME = 'p5-widget.js';
+	var IFRAME_FILENAME = 'p5-widget.html';
+	var IFRAME_STYLE = [
+	    'width: 100%',
+	    'background-color: white',
+	    'border: 1px solid #ec245e',
+	    'box-sizing: border-box'
+	];
+	var AVOID_MIXED_CONTENT_WARNINGS = true;
+	var myScriptEl = getMyScriptEl();
+	var myBaseURL = getMyBaseURL(myScriptEl ? myScriptEl.src : "");
+	var autoload = myScriptEl ? !myScriptEl.hasAttribute('data-manual') : false;
+	var nextId = 1;
+	function getMyBaseURL(url) {
+	    var baseURL = url.slice(0, -MY_FILENAME.length);
+	    if (AVOID_MIXED_CONTENT_WARNINGS) {
+	        if (window.location.protocol === 'http:' && /^https:/.test(baseURL)) {
+	            // Our script was loaded over HTTPS, but the embedding page is
+	            // using HTTP. This is likely to result in mixed content warnings
+	            // if e.g. the widget's sketch wants to load resources relative to
+	            // the embedding page's URL, so let's just embed the widget over
+	            // HTTP instead of HTTPS.
+	            baseURL = baseURL.replace('https:', 'http:');
+	        }
+	    }
+	    return baseURL;
+	}
+	function getMyScriptEl() {
+	    return (document.currentScript ||
+	        document.querySelectorAll("script[src$='" + MY_FILENAME + "']")[0]);
+	}
+	// http://stackoverflow.com/a/7557433/2422398
+	function isElementInViewport(el) {
+	    var rect = el.getBoundingClientRect();
+	    return (rect.bottom >= 0 &&
+	        rect.right >= 0 &&
+	        rect.top <= (window.innerHeight ||
+	            document.documentElement.clientHeight) &&
+	        rect.left <= (window.innerWidth ||
+	            document.documentElement.clientWidth));
+	}
+	function getDataHeight(el) {
+	    var height = parseInt(el.getAttribute('data-height'));
+	    if (isNaN(height))
+	        height = defaults.HEIGHT;
+	    return height;
+	}
+	function absoluteURL(url) {
+	    var a = document.createElement('a');
+	    a.setAttribute('href', url);
+	    return a.href;
+	}
+	function getSketch(url, cb) {
+	    var error = function (msg) {
+	        var lines = ['// p5.js-widget failed to retrieve ' + url + '.'];
+	        if (msg && typeof (msg) == 'string') {
+	            lines.push('// ' + msg);
+	        }
+	        cb(lines.join('\n'));
+	    };
+	    var req = new XMLHttpRequest();
+	    req.open('GET', url);
+	    req.onload = function () {
+	        if (req.status == 200) {
+	            cb(req.responseText);
+	        }
+	        else {
+	            error('Server returned HTTP ' + req.status + '.');
+	        }
+	    };
+	    req.onerror = error;
+	    req.send(null);
+	}
+	function replaceScriptWithWidget(el) {
+	    var iframe = document.createElement('iframe');
+	    var height = getDataHeight(el);
+	    var previewWidth = parseInt(el.getAttribute('data-preview-width'));
+	    var baseSketchURL = absoluteURL(el.getAttribute('data-base-url'));
+	    var p5version = el.getAttribute('data-p5-version');
+	    var maxRunTime = parseInt(el.getAttribute('data-max-run-time'));
+	    var autoplay = el.hasAttribute('data-autoplay');
+	    var url;
+	    var qsArgs = [
+	        'id=' + encodeURIComponent(el.getAttribute('data-id'))
+	    ];
+	    var style = IFRAME_STYLE.slice();
+	    function makeWidget(sketch) {
+	        qsArgs.push('sketch=' + encodeURIComponent(sketch));
+	        style.push('min-height: ' + height + 'px');
+	        url = myBaseURL + IFRAME_FILENAME + '?' + qsArgs.join('&');
+	        iframe.setAttribute('src', url);
+	        iframe.setAttribute('style', style.join('; '));
+	        el.parentNode.replaceChild(iframe, el);
+	    }
+	    if (!isNaN(previewWidth) && previewWidth >= 0) {
+	        qsArgs.push('previewWidth=' + previewWidth);
+	    }
+	    if (!isNaN(maxRunTime) && maxRunTime >= 0) {
+	        qsArgs.push('maxRunTime=' + maxRunTime);
+	    }
+	    if (baseSketchURL) {
+	        qsArgs.push('baseSketchURL=' + encodeURIComponent(baseSketchURL));
+	    }
+	    if (p5version) {
+	        qsArgs.push('p5version=' + encodeURIComponent(p5version));
+	    }
+	    if (autoplay) {
+	        qsArgs.push('autoplay=on');
+	    }
+	    if (el.src && el.textContent && el.textContent.trim()) {
+	        return makeWidget([
+	            '// Your widget includes both a "src" attribute and inline script',
+	            '// content, which makes no sense. Please remove one of them.'
+	        ].join('\n'));
+	    }
+	    if (el.src) {
+	        getSketch(el.src, makeWidget);
+	    }
+	    else {
+	        makeWidget(el.textContent);
+	    }
+	}
+	function whenVisible(el, cb) {
+	    var CHECK_INTERVAL_MS = 1000;
+	    var interval;
+	    function maybeMakeVisible() {
+	        if (!isElementInViewport(el))
+	            return;
+	        clearInterval(interval);
+	        window.removeEventListener('scroll', maybeMakeVisible, false);
+	        window.removeEventListener('resize', maybeMakeVisible, false);
+	        cb(el);
+	    }
+	    // We want to check at a fixed interval as a fallback, to make
+	    // sure that we detect when the element is visible even outside
+	    // of the usual means (e.g., because the user did some
+	    // sort of pinch/zoom gesture).
+	    interval = setInterval(maybeMakeVisible, 1000);
+	    window.addEventListener('scroll', maybeMakeVisible, false);
+	    window.addEventListener('resize', maybeMakeVisible, false);
+	    maybeMakeVisible();
+	}
+	function lazilyReplaceScriptWithWidget(el) {
+	    var height = getDataHeight(el);
+	    el.style.display = 'block';
+	    el.style.fontSize = '0';
+	    el.style.width = '100%';
+	    el.style.minHeight = height + 'px';
+	    el.style.background = '#f0f0f0';
+	    if (!el.hasAttribute('data-id')) {
+	        el.setAttribute('data-id', nextId.toString());
+	        nextId++;
+	    }
+	    whenVisible(el, replaceScriptWithWidget);
+	}
+	function lazilyReplaceAllScriptsWithWidget() {
+	    var scripts = document.querySelectorAll("script[type='text/p5']");
+	    [].slice.call(scripts).forEach(function (el) {
+	        lazilyReplaceScriptWithWidget(el);
+	    });
+	}
+	if (autoload) {
+	    if (document.readyState === 'complete') {
+	        lazilyReplaceAllScriptsWithWidget();
+	    }
+	    else {
+	        window.addEventListener('load', lazilyReplaceAllScriptsWithWidget, false);
+	    }
+	}
+	window['p5Widget'] = {
+	    baseURL: myBaseURL,
+	    url: myBaseURL + MY_FILENAME,
+	    replaceScript: lazilyReplaceScriptWithWidget,
+	    replaceAll: lazilyReplaceAllScriptsWithWidget,
+	    defaults: defaults
+	};
+
+
+/***/ }),
+/* 1 */
+/***/ (function(module, exports) {
+
+	"use strict";
+	exports.P5_VERSION = '0.4.23';
+	exports.PREVIEW_WIDTH = 150;
+	exports.HEIGHT = 300;
+	exports.MAX_RUN_TIME = 1000;
+
+
+/***/ })
+/******/ ]);
+//# sourceMappingURL=p5-widget.js.map

web/p5_export/p5-widget/preview-frame.bundle.js

@@ -0,0 +1,501 @@
+/******/ (function(modules) { // webpackBootstrap
+/******/ 	// The module cache
+/******/ 	var installedModules = {};
+/******/
+/******/ 	// The require function
+/******/ 	function __webpack_require__(moduleId) {
+/******/
+/******/ 		// Check if module is in cache
+/******/ 		if(installedModules[moduleId])
+/******/ 			return installedModules[moduleId].exports;
+/******/
+/******/ 		// Create a new module (and put it into the cache)
+/******/ 		var module = installedModules[moduleId] = {
+/******/ 			exports: {},
+/******/ 			id: moduleId,
+/******/ 			loaded: false
+/******/ 		};
+/******/
+/******/ 		// Execute the module function
+/******/ 		modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
+/******/
+/******/ 		// Flag the module as loaded
+/******/ 		module.loaded = true;
+/******/
+/******/ 		// Return the exports of the module
+/******/ 		return module.exports;
+/******/ 	}
+/******/
+/******/
+/******/ 	// expose the modules object (__webpack_modules__)
+/******/ 	__webpack_require__.m = modules;
+/******/
+/******/ 	// expose the module cache
+/******/ 	__webpack_require__.c = installedModules;
+/******/
+/******/ 	// __webpack_public_path__
+/******/ 	__webpack_require__.p = "";
+/******/
+/******/ 	// Load entry module and return exports
+/******/ 	return __webpack_require__(0);
+/******/ })
+/************************************************************************/
+/******/ ({
+
+/***/ 0:
+/***/ (function(module, exports, __webpack_require__) {
+
+	"use strict";
+	__webpack_require__(216);
+	// @ts-ignore
+	var global = window;
+	function loadScript(url, cb) {
+	    var script = document.createElement('script');
+	    cb = cb || (function () { });
+	    script.onload = cb;
+	    script.onerror = function () {
+	        console.log("Failed to load script: " + url);
+	    };
+	    script.setAttribute('src', url);
+	    document.body.appendChild(script);
+	}
+	function loadScripts(urls, cb) {
+	    cb = cb || (function () { });
+	    var i = 0;
+	    var loadNextScript = function () {
+	        if (i === urls.length) {
+	            return cb();
+	        }
+	        loadScript(urls[i++], loadNextScript);
+	    };
+	    loadNextScript();
+	}
+	function p5url(version) {
+	    return "//cdnjs.cloudflare.com/ajax/libs/p5.js/" + version + "/p5.js";
+	}
+	function LoopChecker(sketch, funcName, maxRunTime) {
+	    var self = {
+	        wasTriggered: false,
+	        getLineNumber: function () {
+	            var index = loopCheckFailureRange[0];
+	            var line = 1;
+	            for (var i = 0; i < index; i++) {
+	                if (sketch[i] === '\n')
+	                    line++;
+	            }
+	            return line;
+	        }
+	    };
+	    var startTime = Date.now();
+	    var loopCheckFailureRange = null;
+	    global[funcName] = function (range) {
+	        if (Date.now() - startTime > maxRunTime) {
+	            self.wasTriggered = true;
+	            loopCheckFailureRange = range;
+	            throw new Error("Loop took over " + maxRunTime + " ms to run");
+	        }
+	    };
+	    setInterval(function () {
+	        startTime = Date.now();
+	    }, maxRunTime / 2);
+	    return self;
+	}
+	function setBaseURL(url) {
+	    var base = document.createElement('base');
+	    base.setAttribute('href', url);
+	    document.head.appendChild(base);
+	}
+	function startSketch(sketch, p5version, maxRunTime, loopCheckFuncName, baseURL, errorCb) {
+	    var sketchScript = document.createElement('script');
+	    var loopChecker = LoopChecker(sketch, loopCheckFuncName, maxRunTime);
+	    if (baseURL) {
+	        setBaseURL(baseURL);
+	    }
+	    sketchScript.textContent = sketch;
+	    global.addEventListener('error', function (e) {
+	        var message = e.message;
+	        var line = undefined;
+	        if (loopChecker.wasTriggered) {
+	            message = "Your loop is taking too long to run.";
+	            line = loopChecker.getLineNumber();
+	        }
+	        else if (typeof (e.lineno) === 'number' &&
+	            (e.filename === '' || e.filename === window.location.href)) {
+	            line = e.lineno;
+	        }
+	        // p5 sketches don't actually stop looping if they throw an exception,
+	        // so try to stop the sketch.
+	        try {
+	            global.noLoop();
+	        }
+	        catch (e) { }
+	        errorCb(message, line);
+	    });
+	    loadScripts([
+	        p5url(p5version),
+	    ], function () {
+	        document.body.appendChild(sketchScript);
+	        if (document.readyState === 'complete') {
+	            new global.p5();
+	        }
+	    });
+	}
+	global.startSketch = startSketch;
+
+
+/***/ }),
+
+/***/ 210:
+/***/ (function(module, exports) {
+
+	/*
+		MIT License http://www.opensource.org/licenses/mit-license.php
+		Author Tobias Koppers @sokra
+	*/
+	// css base code, injected by the css-loader
+	module.exports = function() {
+		var list = [];
+	
+		// return the list of modules as css string
+		list.toString = function toString() {
+			var result = [];
+			for(var i = 0; i < this.length; i++) {
+				var item = this[i];
+				if(item[2]) {
+					result.push("@media " + item[2] + "{" + item[1] + "}");
+				} else {
+					result.push(item[1]);
+				}
+			}
+			return result.join("");
+		};
+	
+		// import a list of modules into the list
+		list.i = function(modules, mediaQuery) {
+			if(typeof modules === "string")
+				modules = [[null, modules, ""]];
+			var alreadyImportedModules = {};
+			for(var i = 0; i < this.length; i++) {
+				var id = this[i][0];
+				if(typeof id === "number")
+					alreadyImportedModules[id] = true;
+			}
+			for(i = 0; i < modules.length; i++) {
+				var item = modules[i];
+				// skip already imported module
+				// this implementation is not 100% perfect for weird media query combinations
+				//  when a module is imported multiple times with different media queries.
+				//  I hope this will never occur (Hey this way we have smaller bundles)
+				if(typeof item[0] !== "number" || !alreadyImportedModules[item[0]]) {
+					if(mediaQuery && !item[2]) {
+						item[2] = mediaQuery;
+					} else if(mediaQuery) {
+						item[2] = "(" + item[2] + ") and (" + mediaQuery + ")";
+					}
+					list.push(item);
+				}
+			}
+		};
+		return list;
+	};
+
+
+/***/ }),
+
+/***/ 211:
+/***/ (function(module, exports, __webpack_require__) {
+
+	/*
+		MIT License http://www.opensource.org/licenses/mit-license.php
+		Author Tobias Koppers @sokra
+	*/
+	var stylesInDom = {},
+		memoize = function(fn) {
+			var memo;
+			return function () {
+				if (typeof memo === "undefined") memo = fn.apply(this, arguments);
+				return memo;
+			};
+		},
+		isOldIE = memoize(function() {
+			return /msie [6-9]\b/.test(self.navigator.userAgent.toLowerCase());
+		}),
+		getHeadElement = memoize(function () {
+			return document.head || document.getElementsByTagName("head")[0];
+		}),
+		singletonElement = null,
+		singletonCounter = 0,
+		styleElementsInsertedAtTop = [];
+	
+	module.exports = function(list, options) {
+		if(false) {
+			if(typeof document !== "object") throw new Error("The style-loader cannot be used in a non-browser environment");
+		}
+	
+		options = options || {};
+		// Force single-tag solution on IE6-9, which has a hard limit on the # of <style>
+		// tags it will allow on a page
+		if (typeof options.singleton === "undefined") options.singleton = isOldIE();
+	
+		// By default, add <style> tags to the bottom of <head>.
+		if (typeof options.insertAt === "undefined") options.insertAt = "bottom";
+	
+		var styles = listToStyles(list);
+		addStylesToDom(styles, options);
+	
+		return function update(newList) {
+			var mayRemove = [];
+			for(var i = 0; i < styles.length; i++) {
+				var item = styles[i];
+				var domStyle = stylesInDom[item.id];
+				domStyle.refs--;
+				mayRemove.push(domStyle);
+			}
+			if(newList) {
+				var newStyles = listToStyles(newList);
+				addStylesToDom(newStyles, options);
+			}
+			for(var i = 0; i < mayRemove.length; i++) {
+				var domStyle = mayRemove[i];
+				if(domStyle.refs === 0) {
+					for(var j = 0; j < domStyle.parts.length; j++)
+						domStyle.parts[j]();
+					delete stylesInDom[domStyle.id];
+				}
+			}
+		};
+	}
+	
+	function addStylesToDom(styles, options) {
+		for(var i = 0; i < styles.length; i++) {
+			var item = styles[i];
+			var domStyle = stylesInDom[item.id];
+			if(domStyle) {
+				domStyle.refs++;
+				for(var j = 0; j < domStyle.parts.length; j++) {
+					domStyle.parts[j](item.parts[j]);
+				}
+				for(; j < item.parts.length; j++) {
+					domStyle.parts.push(addStyle(item.parts[j], options));
+				}
+			} else {
+				var parts = [];
+				for(var j = 0; j < item.parts.length; j++) {
+					parts.push(addStyle(item.parts[j], options));
+				}
+				stylesInDom[item.id] = {id: item.id, refs: 1, parts: parts};
+			}
+		}
+	}
+	
+	function listToStyles(list) {
+		var styles = [];
+		var newStyles = {};
+		for(var i = 0; i < list.length; i++) {
+			var item = list[i];
+			var id = item[0];
+			var css = item[1];
+			var media = item[2];
+			var sourceMap = item[3];
+			var part = {css: css, media: media, sourceMap: sourceMap};
+			if(!newStyles[id])
+				styles.push(newStyles[id] = {id: id, parts: [part]});
+			else
+				newStyles[id].parts.push(part);
+		}
+		return styles;
+	}
+	
+	function insertStyleElement(options, styleElement) {
+		var head = getHeadElement();
+		var lastStyleElementInsertedAtTop = styleElementsInsertedAtTop[styleElementsInsertedAtTop.length - 1];
+		if (options.insertAt === "top") {
+			if(!lastStyleElementInsertedAtTop) {
+				head.insertBefore(styleElement, head.firstChild);
+			} else if(lastStyleElementInsertedAtTop.nextSibling) {
+				head.insertBefore(styleElement, lastStyleElementInsertedAtTop.nextSibling);
+			} else {
+				head.appendChild(styleElement);
+			}
+			styleElementsInsertedAtTop.push(styleElement);
+		} else if (options.insertAt === "bottom") {
+			head.appendChild(styleElement);
+		} else {
+			throw new Error("Invalid value for parameter 'insertAt'. Must be 'top' or 'bottom'.");
+		}
+	}
+	
+	function removeStyleElement(styleElement) {
+		styleElement.parentNode.removeChild(styleElement);
+		var idx = styleElementsInsertedAtTop.indexOf(styleElement);
+		if(idx >= 0) {
+			styleElementsInsertedAtTop.splice(idx, 1);
+		}
+	}
+	
+	function createStyleElement(options) {
+		var styleElement = document.createElement("style");
+		styleElement.type = "text/css";
+		insertStyleElement(options, styleElement);
+		return styleElement;
+	}
+	
+	function createLinkElement(options) {
+		var linkElement = document.createElement("link");
+		linkElement.rel = "stylesheet";
+		insertStyleElement(options, linkElement);
+		return linkElement;
+	}
+	
+	function addStyle(obj, options) {
+		var styleElement, update, remove;
+	
+		if (options.singleton) {
+			var styleIndex = singletonCounter++;
+			styleElement = singletonElement || (singletonElement = createStyleElement(options));
+			update = applyToSingletonTag.bind(null, styleElement, styleIndex, false);
+			remove = applyToSingletonTag.bind(null, styleElement, styleIndex, true);
+		} else if(obj.sourceMap &&
+			typeof URL === "function" &&
+			typeof URL.createObjectURL === "function" &&
+			typeof URL.revokeObjectURL === "function" &&
+			typeof Blob === "function" &&
+			typeof btoa === "function") {
+			styleElement = createLinkElement(options);
+			update = updateLink.bind(null, styleElement);
+			remove = function() {
+				removeStyleElement(styleElement);
+				if(styleElement.href)
+					URL.revokeObjectURL(styleElement.href);
+			};
+		} else {
+			styleElement = createStyleElement(options);
+			update = applyToTag.bind(null, styleElement);
+			remove = function() {
+				removeStyleElement(styleElement);
+			};
+		}
+	
+		update(obj);
+	
+		return function updateStyle(newObj) {
+			if(newObj) {
+				if(newObj.css === obj.css && newObj.media === obj.media && newObj.sourceMap === obj.sourceMap)
+					return;
+				update(obj = newObj);
+			} else {
+				remove();
+			}
+		};
+	}
+	
+	var replaceText = (function () {
+		var textStore = [];
+	
+		return function (index, replacement) {
+			textStore[index] = replacement;
+			return textStore.filter(Boolean).join('\n');
+		};
+	})();
+	
+	function applyToSingletonTag(styleElement, index, remove, obj) {
+		var css = remove ? "" : obj.css;
+	
+		if (styleElement.styleSheet) {
+			styleElement.styleSheet.cssText = replaceText(index, css);
+		} else {
+			var cssNode = document.createTextNode(css);
+			var childNodes = styleElement.childNodes;
+			if (childNodes[index]) styleElement.removeChild(childNodes[index]);
+			if (childNodes.length) {
+				styleElement.insertBefore(cssNode, childNodes[index]);
+			} else {
+				styleElement.appendChild(cssNode);
+			}
+		}
+	}
+	
+	function applyToTag(styleElement, obj) {
+		var css = obj.css;
+		var media = obj.media;
+	
+		if(media) {
+			styleElement.setAttribute("media", media)
+		}
+	
+		if(styleElement.styleSheet) {
+			styleElement.styleSheet.cssText = css;
+		} else {
+			while(styleElement.firstChild) {
+				styleElement.removeChild(styleElement.firstChild);
+			}
+			styleElement.appendChild(document.createTextNode(css));
+		}
+	}
+	
+	function updateLink(linkElement, obj) {
+		var css = obj.css;
+		var sourceMap = obj.sourceMap;
+	
+		if(sourceMap) {
+			// http://stackoverflow.com/a/26603875
+			css += "\n/*# sourceMappingURL=data:application/json;base64," + btoa(unescape(encodeURIComponent(JSON.stringify(sourceMap)))) + " */";
+		}
+	
+		var blob = new Blob([css], { type: "text/css" });
+	
+		var oldSrc = linkElement.href;
+	
+		linkElement.href = URL.createObjectURL(blob);
+	
+		if(oldSrc)
+			URL.revokeObjectURL(oldSrc);
+	}
+
+
+/***/ }),
+
+/***/ 216:
+/***/ (function(module, exports, __webpack_require__) {
+
+	// style-loader: Adds some css to the DOM by adding a <style> tag
+	
+	// load the styles
+	var content = __webpack_require__(217);
+	if(typeof content === 'string') content = [[module.id, content, '']];
+	// add the styles to the DOM
+	var update = __webpack_require__(211)(content, {});
+	if(content.locals) module.exports = content.locals;
+	// Hot Module Replacement
+	if(false) {
+		// When the styles change, update the <style> tags
+		if(!content.locals) {
+			module.hot.accept("!!../node_modules/.store/css-loader@0.23.1/node_modules/css-loader/index.js?sourceMap!../node_modules/.store/postcss-loader@0.8.2/node_modules/postcss-loader/index.js?sourceMap!./preview-frame.css", function() {
+				var newContent = require("!!../node_modules/.store/css-loader@0.23.1/node_modules/css-loader/index.js?sourceMap!../node_modules/.store/postcss-loader@0.8.2/node_modules/postcss-loader/index.js?sourceMap!./preview-frame.css");
+				if(typeof newContent === 'string') newContent = [[module.id, newContent, '']];
+				update(newContent);
+			});
+		}
+		// When the module is disposed, remove the <style> tags
+		module.hot.dispose(function() { update(); });
+	}
+
+/***/ }),
+
+/***/ 217:
+/***/ (function(module, exports, __webpack_require__) {
+
+	exports = module.exports = __webpack_require__(210)();
+	// imports
+	
+	
+	// module
+	exports.push([module.id, "html, body {\r\n  height: 100%;\r\n}\r\n\r\nbody {\r\n  margin: 0;\r\n  display: -ms-flexbox;\r\n  display: flex;\r\n\r\n  /* This centers our sketch horizontally. */\r\n  -ms-flex-pack: center;\r\n      justify-content: center;\r\n\r\n  /* This centers our sketch vertically. */\r\n  -ms-flex-align: center;\r\n      align-items: center;\r\n}\r\n", "", {"version":3,"sources":["/./css/preview-frame.css"],"names":[],"mappings":"AAAA;EACE,aAAa;CACd;;AAED;EACE,UAAU;EACV,qBAAc;EAAd,cAAc;;EAEd,2CAA2C;EAC3C,sBAAwB;MAAxB,wBAAwB;;EAExB,yCAAyC;EACzC,uBAAoB;MAApB,oBAAoB;CACrB","file":"preview-frame.css","sourcesContent":["html, body {\r\n  height: 100%;\r\n}\r\n\r\nbody {\r\n  margin: 0;\r\n  display: flex;\r\n\r\n  /* This centers our sketch horizontally. */\r\n  justify-content: center;\r\n\r\n  /* This centers our sketch vertically. */\r\n  align-items: center;\r\n}\r\n"],"sourceRoot":"webpack://"}]);
+	
+	// exports
+
+
+/***/ })
+
+/******/ });
+//# sourceMappingURL=preview-frame.bundle.js.map

web/p5_export/p5-widget/preview-frame.html

@@ -0,0 +1,43 @@
+<!DOCTYPE html>
+<meta charset="utf-8">
+<title>Preview</title>
+
+<body>
+  <script src="./p5.js"></script>
+
+  <script src="./src/CCapture.js"></script>
+  <!-- <script src="src/gif.js"></script> -->
+  <!-- <script src="src/gif.worker.js"></script> -->
+  <!-- <script src="src/download.js"></script> -->
+  <!-- <script src="src/tar.js"></script> -->
+
+  <script>
+    let capturer = new CCapture({
+      format: 'png',
+      framerate: 60,
+      verbose: true
+    });
+
+
+    var capturer_start = () => {
+      if (frameCount === 1) {
+        capturer.start();
+      }
+    }
+
+
+    var capturer_end = (t = 24) => {
+
+      if (frameCount < t) {
+        capturer.capture(canvas, t);
+      } else if (frameCount === t) {
+        capturer.save(null);
+        capturer.stop();
+        capturer = null;
+      }
+    }
+
+  </script>
+
+  <script src="./preview-frame.bundle.js"></script>
+</body>

web/p5_export/p5-widget/src/Whammy.js

@@ -0,0 +1,585 @@
+/*
+	var vid = new Whammy.Video();
+	vid.add(canvas or data url)
+	vid.compile()
+*/
+
+window.Whammy = (function(){
+	// in this case, frames has a very specific meaning, which will be
+	// detailed once i finish writing the code
+
+	function toWebM(frames, outputAsArray){
+		var info = checkFrames(frames);
+
+		//max duration by cluster in milliseconds
+		var CLUSTER_MAX_DURATION = 30000;
+
+		var EBML = [
+			{
+				"id": 0x1a45dfa3, // EBML
+				"data": [
+					{
+						"data": 1,
+						"id": 0x4286 // EBMLVersion
+					},
+					{
+						"data": 1,
+						"id": 0x42f7 // EBMLReadVersion
+					},
+					{
+						"data": 4,
+						"id": 0x42f2 // EBMLMaxIDLength
+					},
+					{
+						"data": 8,
+						"id": 0x42f3 // EBMLMaxSizeLength
+					},
+					{
+						"data": "webm",
+						"id": 0x4282 // DocType
+					},
+					{
+						"data": 2,
+						"id": 0x4287 // DocTypeVersion
+					},
+					{
+						"data": 2,
+						"id": 0x4285 // DocTypeReadVersion
+					}
+				]
+			},
+			{
+				"id": 0x18538067, // Segment
+				"data": [
+					{
+						"id": 0x1549a966, // Info
+						"data": [
+							{
+								"data": 1e6, //do things in millisecs (num of nanosecs for duration scale)
+								"id": 0x2ad7b1 // TimecodeScale
+							},
+							{
+								"data": "whammy",
+								"id": 0x4d80 // MuxingApp
+							},
+							{
+								"data": "whammy",
+								"id": 0x5741 // WritingApp
+							},
+							{
+								"data": doubleToString(info.duration),
+								"id": 0x4489 // Duration
+							}
+						]
+					},
+					{
+						"id": 0x1654ae6b, // Tracks
+						"data": [
+							{
+								"id": 0xae, // TrackEntry
+								"data": [
+									{
+										"data": 1,
+										"id": 0xd7 // TrackNumber
+									},
+									{
+										"data": 1,
+										"id": 0x73c5 // TrackUID
+									},
+									{
+										"data": 0,
+										"id": 0x9c // FlagLacing
+									},
+									{
+										"data": "und",
+										"id": 0x22b59c // Language
+									},
+									{
+										"data": "V_VP8",
+										"id": 0x86 // CodecID
+									},
+									{
+										"data": "VP8",
+										"id": 0x258688 // CodecName
+									},
+									{
+										"data": 1,
+										"id": 0x83 // TrackType
+									},
+									{
+										"id": 0xe0,  // Video
+										"data": [
+											{
+												"data": info.width,
+												"id": 0xb0 // PixelWidth
+											},
+											{
+												"data": info.height,
+												"id": 0xba // PixelHeight
+											}
+										]
+									}
+								]
+							}
+						]
+					},
+					{
+						"id": 0x1c53bb6b, // Cues
+						"data": [
+							//cue insertion point
+						]
+					}
+
+					//cluster insertion point
+				]
+			}
+		 ];
+
+
+		var segment = EBML[1];
+		var cues = segment.data[2];
+
+		//Generate clusters (max duration)
+		var frameNumber = 0;
+		var clusterTimecode = 0;
+		while(frameNumber < frames.length){
+
+			var cuePoint = {
+					"id": 0xbb, // CuePoint
+					"data": [
+						{
+							"data": Math.round(clusterTimecode),
+							"id": 0xb3 // CueTime
+						},
+						{
+							"id": 0xb7, // CueTrackPositions
+							"data": [
+								{
+									"data": 1,
+									"id": 0xf7 // CueTrack
+								},
+								{
+									"data": 0, // to be filled in when we know it
+									"size": 8,
+									"id": 0xf1 // CueClusterPosition
+								}
+							]
+						}
+					]
+				};
+
+			cues.data.push(cuePoint);
+
+			var clusterFrames = [];
+			var clusterDuration = 0;
+			do {
+				clusterFrames.push(frames[frameNumber]);
+				clusterDuration += frames[frameNumber].duration;
+				frameNumber++;
+			}while(frameNumber < frames.length && clusterDuration < CLUSTER_MAX_DURATION);
+
+			var clusterCounter = 0;
+			var cluster = {
+					"id": 0x1f43b675, // Cluster
+					"data": [
+						{
+							"data": Math.round(clusterTimecode),
+							"id": 0xe7 // Timecode
+						}
+					].concat(clusterFrames.map(function(webp){
+						var block = makeSimpleBlock({
+							discardable: 0,
+							frame: webp.data.slice(4),
+							invisible: 0,
+							keyframe: 1,
+							lacing: 0,
+							trackNum: 1,
+							timecode: Math.round(clusterCounter)
+						});
+						clusterCounter += webp.duration;
+						return {
+							data: block,
+							id: 0xa3
+						};
+					}))
+				}
+
+			//Add cluster to segment
+			segment.data.push(cluster);
+			clusterTimecode += clusterDuration;
+		}
+
+		//First pass to compute cluster positions
+		var position = 0;
+		for(var i = 0; i < segment.data.length; i++){
+			if (i >= 3) {
+				cues.data[i-3].data[1].data[1].data = position;
+			}
+			var data = generateEBML([segment.data[i]], outputAsArray);
+			position += data.size || data.byteLength || data.length;
+			if (i != 2) { // not cues
+				//Save results to avoid having to encode everything twice
+				segment.data[i] = data;
+			}
+		}
+
+		return generateEBML(EBML, outputAsArray)
+	}
+
+	// sums the lengths of all the frames and gets the duration, woo
+
+	function checkFrames(frames){
+		var width = frames[0].width,
+			height = frames[0].height,
+			duration = frames[0].duration;
+		for(var i = 1; i < frames.length; i++){
+			if(frames[i].width != width) throw "Frame " + (i + 1) + " has a different width";
+			if(frames[i].height != height) throw "Frame " + (i + 1) + " has a different height";
+			if(frames[i].duration < 0 || frames[i].duration > 0x7fff) throw "Frame " + (i + 1) + " has a weird duration (must be between 0 and 32767)";
+			duration += frames[i].duration;
+		}
+		return {
+			duration: duration,
+			width: width,
+			height: height
+		};
+	}
+
+
+	function numToBuffer(num){
+		var parts = [];
+		while(num > 0){
+			parts.push(num & 0xff)
+			num = num >> 8
+		}
+		return new Uint8Array(parts.reverse());
+	}
+
+	function numToFixedBuffer(num, size){
+		var parts = new Uint8Array(size);
+		for(var i = size - 1; i >= 0; i--){
+			parts[i] = num & 0xff;
+			num = num >> 8;
+		}
+		return parts;
+	}
+
+	function strToBuffer(str){
+		// return new Blob([str]);
+
+		var arr = new Uint8Array(str.length);
+		for(var i = 0; i < str.length; i++){
+			arr[i] = str.charCodeAt(i)
+		}
+		return arr;
+		// this is slower
+		// return new Uint8Array(str.split('').map(function(e){
+		// 	return e.charCodeAt(0)
+		// }))
+	}
+
+
+	//sorry this is ugly, and sort of hard to understand exactly why this was done
+	// at all really, but the reason is that there's some code below that i dont really
+	// feel like understanding, and this is easier than using my brain.
+
+	function bitsToBuffer(bits){
+		var data = [];
+		var pad = (bits.length % 8) ? (new Array(1 + 8 - (bits.length % 8))).join('0') : '';
+		bits = pad + bits;
+		for(var i = 0; i < bits.length; i+= 8){
+			data.push(parseInt(bits.substr(i,8),2))
+		}
+		return new Uint8Array(data);
+	}
+
+	function generateEBML(json, outputAsArray){
+		var ebml = [];
+		for(var i = 0; i < json.length; i++){
+			if (!('id' in json[i])){
+				//already encoded blob or byteArray
+				ebml.push(json[i]);
+				continue;
+			}
+
+			var data = json[i].data;
+			if(typeof data == 'object') data = generateEBML(data, outputAsArray);
+			if(typeof data == 'number') data = ('size' in json[i]) ? numToFixedBuffer(data, json[i].size) : bitsToBuffer(data.toString(2));
+			if(typeof data == 'string') data = strToBuffer(data);
+
+			if(data.length){
+				var z = z;
+			}
+
+			/*var len = data.size || data.byteLength || data.length;
+			var zeroes = Math.ceil(Math.ceil(Math.log(len)/Math.log(2))/8);
+			var size_str = len.toString(2);
+			var padded = (new Array((zeroes * 7 + 7 + 1) - size_str.length)).join('0') + size_str;
+			var size = (new Array(zeroes)).join('0') + '1' + padded;*/
+
+			var len = data.size || data.byteLength || data.length;
+			var zeroes = 0
+			for( var j = 56; j > 0; j-= 7 ) {
+				if( len > Math.pow( 2, j ) - 2 ) {
+					zeroes = j / 7
+					break
+				}
+			}
+			var size_str = len.toString(2);
+			var base = ( new Array( 8 * ( zeroes + 1 ) + 1 ) ).join( '0' )
+			var pre = ( new Array( zeroes + 1 ) ).join( '0' ) + 1;
+			var padded = base.substr( 0, base.length - size_str.length - pre.length ) + size_str;
+			var size = pre + padded;
+			//i actually dont quite understand what went on up there, so I'm not really
+			//going to fix this, i'm probably just going to write some hacky thing which
+			//converts that string into a buffer-esque thing
+
+			ebml.push(numToBuffer(json[i].id));
+			ebml.push(bitsToBuffer(size));
+			ebml.push(data)
+
+
+		}
+
+		//output as blob or byteArray
+		if(outputAsArray){
+			//convert ebml to an array
+			var buffer = toFlatArray(ebml)
+			return new Uint8Array(buffer);
+		}else{
+			return new Blob(ebml, {type: "video/webm"});
+		}
+	}
+
+	function toFlatArray(arr, outBuffer){
+		if(outBuffer == null){
+			outBuffer = [];
+		}
+		for(var i = 0; i < arr.length; i++){
+			if(typeof arr[i] == 'object'){
+				//an array
+				toFlatArray(arr[i], outBuffer)
+			}else{
+				//a simple element
+				outBuffer.push(arr[i]);
+			}
+		}
+		return outBuffer;
+	}
+
+	//OKAY, so the following two functions are the string-based old stuff, the reason they're
+	//still sort of in here, is that they're actually faster than the new blob stuff because
+	//getAsFile isn't widely implemented, or at least, it doesn't work in chrome, which is the
+	// only browser which supports get as webp
+
+	//Converting between a string of 0010101001's and binary back and forth is probably inefficient
+	//TODO: get rid of this function
+	function toBinStr_old(bits){
+		var data = '';
+		var pad = (bits.length % 8) ? (new Array(1 + 8 - (bits.length % 8))).join('0') : '';
+		bits = pad + bits;
+		for(var i = 0; i < bits.length; i+= 8){
+			data += String.fromCharCode(parseInt(bits.substr(i,8),2))
+		}
+		return data;
+	}
+
+	function generateEBML_old(json){
+		var ebml = '';
+		for(var i = 0; i < json.length; i++){
+			var data = json[i].data;
+			if(typeof data == 'object') data = generateEBML_old(data);
+			if(typeof data == 'number') data = toBinStr_old(data.toString(2));
+
+			var len = data.length;
+			var zeroes = Math.ceil(Math.ceil(Math.log(len)/Math.log(2))/8);
+			var size_str = len.toString(2);
+			var padded = (new Array((zeroes * 7 + 7 + 1) - size_str.length)).join('0') + size_str;
+			var size = (new Array(zeroes)).join('0') + '1' + padded;
+
+			ebml += toBinStr_old(json[i].id.toString(2)) + toBinStr_old(size) + data;
+
+		}
+		return ebml;
+	}
+
+	//woot, a function that's actually written for this project!
+	//this parses some json markup and makes it into that binary magic
+	//which can then get shoved into the matroska comtainer (peaceably)
+
+	function makeSimpleBlock(data){
+		var flags = 0;
+		if (data.keyframe) flags |= 128;
+		if (data.invisible) flags |= 8;
+		if (data.lacing) flags |= (data.lacing << 1);
+		if (data.discardable) flags |= 1;
+		if (data.trackNum > 127) {
+			throw "TrackNumber > 127 not supported";
+		}
+		var out = [data.trackNum | 0x80, data.timecode >> 8, data.timecode & 0xff, flags].map(function(e){
+			return String.fromCharCode(e)
+		}).join('') + data.frame;
+
+		return out;
+	}
+
+	// here's something else taken verbatim from weppy, awesome rite?
+
+	function parseWebP(riff){
+		var VP8 = riff.RIFF[0].WEBP[0];
+
+		var frame_start = VP8.indexOf('\x9d\x01\x2a'); //A VP8 keyframe starts with the 0x9d012a header
+		for(var i = 0, c = []; i < 4; i++) c[i] = VP8.charCodeAt(frame_start + 3 + i);
+
+		var width, horizontal_scale, height, vertical_scale, tmp;
+
+		//the code below is literally copied verbatim from the bitstream spec
+		tmp = (c[1] << 8) | c[0];
+		width = tmp & 0x3FFF;
+		horizontal_scale = tmp >> 14;
+		tmp = (c[3] << 8) | c[2];
+		height = tmp & 0x3FFF;
+		vertical_scale = tmp >> 14;
+		return {
+			width: width,
+			height: height,
+			data: VP8,
+			riff: riff
+		}
+	}
+
+	// i think i'm going off on a riff by pretending this is some known
+	// idiom which i'm making a casual and brilliant pun about, but since
+	// i can't find anything on google which conforms to this idiomatic
+	// usage, I'm assuming this is just a consequence of some psychotic
+	// break which makes me make up puns. well, enough riff-raff (aha a
+	// rescue of sorts), this function was ripped wholesale from weppy
+
+	function parseRIFF(string){
+		var offset = 0;
+		var chunks = {};
+
+		while (offset < string.length) {
+			var id = string.substr(offset, 4);
+			chunks[id] = chunks[id] || [];
+			if (id == 'RIFF' || id == 'LIST') {
+				var len = parseInt(string.substr(offset + 4, 4).split('').map(function(i){
+					var unpadded = i.charCodeAt(0).toString(2);
+					return (new Array(8 - unpadded.length + 1)).join('0') + unpadded
+				}).join(''),2);
+				var data = string.substr(offset + 4 + 4, len);
+				offset += 4 + 4 + len;
+				chunks[id].push(parseRIFF(data));
+			} else if (id == 'WEBP') {
+				// Use (offset + 8) to skip past "VP8 "/"VP8L"/"VP8X" field after "WEBP"
+				chunks[id].push(string.substr(offset + 8));
+				offset = string.length;
+			} else {
+				// Unknown chunk type; push entire payload
+				chunks[id].push(string.substr(offset + 4));
+				offset = string.length;
+			}
+		}
+		return chunks;
+	}
+
+	// here's a little utility function that acts as a utility for other functions
+	// basically, the only purpose is for encoding "Duration", which is encoded as
+	// a double (considerably more difficult to encode than an integer)
+	function doubleToString(num){
+		return [].slice.call(
+			new Uint8Array(
+				(
+					new Float64Array([num]) //create a float64 array
+				).buffer) //extract the array buffer
+			, 0) // convert the Uint8Array into a regular array
+			.map(function(e){ //since it's a regular array, we can now use map
+				return String.fromCharCode(e) // encode all the bytes individually
+			})
+			.reverse() //correct the byte endianness (assume it's little endian for now)
+			.join('') // join the bytes in holy matrimony as a string
+	}
+
+	function WhammyVideo(speed, quality){ // a more abstract-ish API
+		this.frames = [];
+		this.duration = 1000 / speed;
+		this.quality = quality || 0.8;
+	}
+
+	WhammyVideo.prototype.add = function(frame, duration){
+		if(typeof duration != 'undefined' && this.duration) throw "you can't pass a duration if the fps is set";
+		if(typeof duration == 'undefined' && !this.duration) throw "if you don't have the fps set, you need to have durations here.";
+		if(frame.canvas){ //CanvasRenderingContext2D
+			frame = frame.canvas;
+		}
+		if(frame.toDataURL){
+			// frame = frame.toDataURL('image/webp', this.quality);
+			// quickly store image data so we don't block cpu. encode in compile method.
+			frame = frame.getContext('2d').getImageData(0, 0, frame.width, frame.height);
+		}else if(typeof frame != "string"){
+			throw "frame must be a a HTMLCanvasElement, a CanvasRenderingContext2D or a DataURI formatted string"
+		}
+		if (typeof frame === "string" && !(/^data:image\/webp;base64,/ig).test(frame)) {
+			throw "Input must be formatted properly as a base64 encoded DataURI of type image/webp";
+		}
+		this.frames.push({
+			image: frame,
+			duration: duration || this.duration
+		});
+	};
+
+	// deferred webp encoding. Draws image data to canvas, then encodes as dataUrl
+	WhammyVideo.prototype.encodeFrames = function(callback){
+
+		if(this.frames[0].image instanceof ImageData){
+
+			var frames = this.frames;
+			var tmpCanvas = document.createElement('canvas');
+			var tmpContext = tmpCanvas.getContext('2d');
+			tmpCanvas.width = this.frames[0].image.width;
+			tmpCanvas.height = this.frames[0].image.height;
+
+			var encodeFrame = function(index){
+				var frame = frames[index];
+				tmpContext.putImageData(frame.image, 0, 0);
+				frame.image = tmpCanvas.toDataURL('image/webp', this.quality);
+				if(index < frames.length-1){
+					setTimeout(function(){ encodeFrame(index + 1); }, 1);
+				}else{
+					callback();
+				}
+			}.bind(this);
+
+			encodeFrame(0);
+		}else{
+			callback();
+		}
+	};
+
+	WhammyVideo.prototype.compile = function(outputAsArray, callback){
+
+		this.encodeFrames(function(){
+
+			var webm = new toWebM(this.frames.map(function(frame){
+				var webp = parseWebP(parseRIFF(atob(frame.image.slice(23))));
+				webp.duration = frame.duration;
+				return webp;
+			}), outputAsArray);
+			callback(webm);
+			
+		}.bind(this));
+	};
+
+	return {
+		Video: WhammyVideo,
+		fromImageArray: function(images, fps, outputAsArray){
+			return toWebM(images.map(function(image){
+				var webp = parseWebP(parseRIFF(atob(image.slice(23))))
+				webp.duration = 1000 / fps;
+				return webp;
+			}), outputAsArray)
+		},
+		toWebM: toWebM
+		// expose methods of madness
+	}
+})()

web/p5_export/p5-widget/src/tar.js

@@ -0,0 +1,334 @@
+(function () {
+	"use strict";
+
+	var lookup = [
+			'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H',
+			'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P',
+			'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X',
+			'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f',
+			'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',
+			'o', 'p', 'q', 'r', 's', 't', 'u', 'v',
+			'w', 'x', 'y', 'z', '0', '1', '2', '3',
+			'4', '5', '6', '7', '8', '9', '+', '/'
+		];
+	function clean(length) {
+		var i, buffer = new Uint8Array(length);
+		for (i = 0; i < length; i += 1) {
+			buffer[i] = 0;
+		}
+		return buffer;
+	}
+
+	function extend(orig, length, addLength, multipleOf) {
+		var newSize = length + addLength,
+			buffer = clean((parseInt(newSize / multipleOf) + 1) * multipleOf);
+
+		buffer.set(orig);
+
+		return buffer;
+	}
+
+	function pad(num, bytes, base) {
+		num = num.toString(base || 8);
+		return "000000000000".substr(num.length + 12 - bytes) + num;
+	}	
+	
+	function stringToUint8 (input, out, offset) {
+		var i, length;
+
+		out = out || clean(input.length);
+
+		offset = offset || 0;
+		for (i = 0, length = input.length; i < length; i += 1) {
+			out[offset] = input.charCodeAt(i);
+			offset += 1;
+		}
+
+		return out;
+	}
+
+	function uint8ToBase64(uint8) {
+		var i,
+			extraBytes = uint8.length % 3, // if we have 1 byte left, pad 2 bytes
+			output = "",
+			temp, length;
+
+		function tripletToBase64 (num) {
+			return lookup[num >> 18 & 0x3F] + lookup[num >> 12 & 0x3F] + lookup[num >> 6 & 0x3F] + lookup[num & 0x3F];
+		};
+
+		// go through the array every three bytes, we'll deal with trailing stuff later
+		for (i = 0, length = uint8.length - extraBytes; i < length; i += 3) {
+			temp = (uint8[i] << 16) + (uint8[i + 1] << 8) + (uint8[i + 2]);
+			output += tripletToBase64(temp);
+		}
+
+		// this prevents an ERR_INVALID_URL in Chrome (Firefox okay)
+		switch (output.length % 4) {
+			case 1:
+				output += '=';
+				break;
+			case 2:
+				output += '==';
+				break;
+			default:
+				break;
+		}
+
+		return output;
+	}
+
+	window.utils = {}
+	window.utils.clean = clean;
+	window.utils.pad = pad;
+	window.utils.extend = extend;
+	window.utils.stringToUint8 = stringToUint8;
+	window.utils.uint8ToBase64 = uint8ToBase64;
+}());
+
+(function () {
+	"use strict";
+	
+/*
+struct posix_header {             // byte offset
+	char name[100];               //   0
+	char mode[8];                 // 100
+	char uid[8];                  // 108
+	char gid[8];                  // 116
+	char size[12];                // 124
+	char mtime[12];               // 136
+	char chksum[8];               // 148
+	char typeflag;                // 156
+	char linkname[100];           // 157
+	char magic[6];                // 257
+	char version[2];              // 263
+	char uname[32];               // 265
+	char gname[32];               // 297
+	char devmajor[8];             // 329
+	char devminor[8];             // 337
+	char prefix[155];             // 345
+                                  // 500
+};
+*/
+
+	var utils = window.utils,
+		headerFormat;
+	
+	headerFormat = [
+		{
+			'field': 'fileName',
+			'length': 100
+		},
+		{
+			'field': 'fileMode',
+			'length': 8
+		},
+		{
+			'field': 'uid',
+			'length': 8
+		},
+		{
+			'field': 'gid',
+			'length': 8
+		},
+		{
+			'field': 'fileSize',
+			'length': 12
+		},
+		{
+			'field': 'mtime',
+			'length': 12
+		},
+		{
+			'field': 'checksum',
+			'length': 8
+		},
+		{
+			'field': 'type',
+			'length': 1
+		},
+		{
+			'field': 'linkName',
+			'length': 100
+		},
+		{
+			'field': 'ustar',
+			'length': 8
+		},
+		{
+			'field': 'owner',
+			'length': 32
+		},
+		{
+			'field': 'group',
+			'length': 32
+		},
+		{
+			'field': 'majorNumber',
+			'length': 8
+		},
+		{
+			'field': 'minorNumber',
+			'length': 8
+		},
+		{
+			'field': 'filenamePrefix',
+			'length': 155
+		},
+		{
+			'field': 'padding',
+			'length': 12
+		}
+	];
+
+	function formatHeader(data, cb) {
+		var buffer = utils.clean(512),
+			offset = 0;
+
+		headerFormat.forEach(function (value) {
+			var str = data[value.field] || "",
+				i, length;
+
+			for (i = 0, length = str.length; i < length; i += 1) {
+				buffer[offset] = str.charCodeAt(i);
+				offset += 1;
+			}
+
+			offset += value.length - i; // space it out with nulls
+		});
+
+		if (typeof cb === 'function') {
+			return cb(buffer, offset);
+		}
+		return buffer;
+	}
+	
+	window.header = {}
+	window.header.structure = headerFormat;
+	window.header.format = formatHeader;
+}());
+
+(function () {
+	"use strict";
+
+	var header = window.header,
+		utils = window.utils,
+		recordSize = 512,
+		blockSize;
+	
+	function Tar(recordsPerBlock) {
+		this.written = 0;
+		blockSize = (recordsPerBlock || 20) * recordSize;
+		this.out = utils.clean(blockSize);
+		this.blocks = [];
+		this.length = 0;
+	}
+
+	Tar.prototype.append = function (filepath, input, opts, callback) {
+		var data,
+			checksum,
+			mode,
+			mtime,
+			uid,
+			gid,
+			headerArr;
+
+		if (typeof input === 'string') {
+			input = utils.stringToUint8(input);
+		} else if (input.constructor !== Uint8Array.prototype.constructor) {
+			throw 'Invalid input type. You gave me: ' + input.constructor.toString().match(/function\s*([$A-Za-z_][0-9A-Za-z_]*)\s*\(/)[1];
+		}
+
+		if (typeof opts === 'function') {
+			callback = opts;
+			opts = {};
+		}
+
+		opts = opts || {};
+
+		mode = opts.mode || parseInt('777', 8) & 0xfff;
+		mtime = opts.mtime || Math.floor(+new Date() / 1000);
+		uid = opts.uid || 0;
+		gid = opts.gid || 0;
+
+		data = {
+			fileName: filepath,
+			fileMode: utils.pad(mode, 7),
+			uid: utils.pad(uid, 7),
+			gid: utils.pad(gid, 7),
+			fileSize: utils.pad(input.length, 11),
+			mtime: utils.pad(mtime, 11),
+			checksum: '        ',
+			type: '0', // just a file
+			ustar: 'ustar  ',
+			owner: opts.owner || '',
+			group: opts.group || ''
+		};
+
+		// calculate the checksum
+		checksum = 0;
+		Object.keys(data).forEach(function (key) {
+			var i, value = data[key], length;
+
+			for (i = 0, length = value.length; i < length; i += 1) {
+				checksum += value.charCodeAt(i);
+			}
+		});
+
+		data.checksum = utils.pad(checksum, 6) + "\u0000 ";
+
+		headerArr = header.format(data);
+
+		var headerLength = Math.ceil( headerArr.length / recordSize ) * recordSize; 
+		var inputLength = Math.ceil( input.length / recordSize ) * recordSize;
+
+		this.blocks.push( { header: headerArr, input: input, headerLength: headerLength, inputLength: inputLength } );
+
+	};
+
+	Tar.prototype.save = function() {
+
+		var buffers = [];
+		var chunks = [];
+		var length = 0;
+		var max = Math.pow( 2, 20 );
+
+		var chunk = [];
+		this.blocks.forEach( function( b ) {
+			if( length + b.headerLength + b.inputLength > max ) {
+				chunks.push( { blocks: chunk, length: length } );
+				chunk = [];
+				length = 0;
+			}
+			chunk.push( b );
+			length += b.headerLength + b.inputLength;
+		} );
+		chunks.push( { blocks: chunk, length: length } );
+
+		chunks.forEach( function( c ) {
+
+			var buffer = new Uint8Array( c.length );
+			var written = 0;
+			c.blocks.forEach( function( b ) {
+				buffer.set( b.header, written );
+				written += b.headerLength;
+				buffer.set( b.input, written );
+				written += b.inputLength;
+			} );
+			buffers.push( buffer );
+
+		} );
+
+		buffers.push( new Uint8Array( 2 * recordSize ) );
+
+		return new Blob( buffers, { type: 'octet/stream' } );
+		
+	};
+
+	Tar.prototype.clear = function () {
+		this.written = 0;
+		this.out = utils.clean(blockSize);
+	};
+
+	window.Tar = Tar;
+}());

web/p5_export/p5-widget/static/img/p5js-beta.svg

@@ -0,0 +1,54 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 16.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+   width="250px" height="114px" viewBox="0 0 250 114" enable-background="new 0 0 250 114" xml:space="preserve">
+<path fill="#EC245E" d="M16.254,27.631v7.998h0.359c0.715-1.113,1.65-2.248,2.805-3.402c1.155-1.154,2.568-2.188,4.24-3.105
+  c1.67-0.912,3.561-1.67,5.67-2.268c2.107-0.596,4.477-0.896,7.104-0.896c4.059,0,7.799,0.777,11.223,2.328
+  c3.422,1.555,6.367,3.684,8.836,6.389c2.465,2.707,4.375,5.891,5.73,9.551c1.352,3.662,2.029,7.602,2.029,11.82
+  s-0.656,8.179-1.971,11.879c-1.312,3.701-3.184,6.925-5.611,9.67c-2.427,2.746-5.371,4.938-8.834,6.566
+  c-3.463,1.631-7.385,2.446-11.76,2.446c-4.061,0-7.781-0.836-11.164-2.506c-3.385-1.672-5.99-3.938-7.82-6.807h-0.238v36.295H2.525
+  V27.631H16.254z M49.684,56.045c0-2.229-0.338-4.438-1.014-6.627c-0.678-2.188-1.693-4.158-3.045-5.91
+  c-1.354-1.748-3.064-3.162-5.135-4.238c-2.07-1.074-4.496-1.611-7.281-1.611c-2.627,0-4.977,0.557-7.045,1.672
+  c-2.07,1.115-3.842,2.549-5.312,4.297c-1.475,1.752-2.588,3.742-3.344,5.971c-0.758,2.229-1.133,4.459-1.133,6.686
+  c0,2.229,0.375,4.438,1.133,6.625c0.756,2.191,1.869,4.16,3.344,5.912c1.471,1.75,3.242,3.164,5.312,4.236
+  c2.068,1.075,4.418,1.61,7.045,1.61c2.785,0,5.211-0.555,7.281-1.67c2.07-1.115,3.781-2.547,5.135-4.299
+  c1.352-1.75,2.367-3.74,3.045-5.97C49.346,60.502,49.684,58.273,49.684,56.045z M189.332,24.893v63.505
+  c0,3.422-0.279,6.666-0.836,9.73c-0.559,3.064-1.611,5.73-3.164,8c-1.551,2.27-3.662,4.078-6.328,5.432
+  c-2.668,1.354-6.146,2.029-10.445,2.029c-1.193,0-2.389-0.08-3.582-0.238c-1.193-0.16-2.148-0.319-2.865-0.479l1.195-12.178
+  c0.637,0.16,1.311,0.279,2.027,0.359c0.717,0.077,1.354,0.118,1.91,0.118c1.67,0,3.023-0.317,4.059-0.955
+  c1.033-0.639,1.83-1.514,2.391-2.627c0.555-1.114,0.914-2.407,1.074-3.881c0.156-1.474,0.236-3.043,0.236-4.715V24.893H189.332z
+   M238.162,42.912c-1.275-1.672-3.025-3.123-5.254-4.357c-2.229-1.234-4.656-1.852-7.283-1.852c-2.309,0-4.416,0.479-6.326,1.434
+  c-1.912,0.953-2.863,2.547-2.863,4.775s1.053,3.803,3.16,4.715c2.109,0.916,5.195,1.852,9.256,2.807
+  c2.146,0.479,4.314,1.115,6.506,1.91c2.189,0.795,4.18,1.85,5.971,3.164c1.789,1.312,3.242,2.945,4.357,4.895
+  c1.111,1.951,1.672,4.318,1.672,7.104c0,3.504-0.658,6.47-1.973,8.896c-1.311,2.428-3.062,4.397-5.254,5.91
+  c-2.189,1.512-4.734,2.606-7.641,3.283c-2.906,0.676-5.908,1.014-9.014,1.014c-4.459,0-8.795-0.816-13.014-2.447
+  c-4.219-1.629-7.721-3.959-10.506-6.982l9.432-8.836c1.592,2.07,3.66,3.781,6.209,5.133c2.547,1.354,5.371,2.029,8.477,2.029
+  c1.033,0,2.088-0.117,3.164-0.357c1.074-0.237,2.068-0.614,2.984-1.133c0.914-0.518,1.65-1.213,2.209-2.09
+  c0.555-0.877,0.834-1.949,0.834-3.225c0-2.389-1.094-4.098-3.281-5.133c-2.191-1.035-5.475-2.07-9.85-3.104
+  c-2.15-0.479-4.24-1.094-6.27-1.853c-2.029-0.756-3.84-1.75-5.432-2.983c-1.596-1.234-2.865-2.764-3.82-4.598
+  c-0.955-1.83-1.436-4.098-1.436-6.805c0-3.184,0.656-5.928,1.973-8.236c1.311-2.312,3.045-4.197,5.191-5.674
+  c2.148-1.471,4.576-2.566,7.283-3.281c2.705-0.717,5.492-1.076,8.357-1.076c4.137,0,8.178,0.717,12.117,2.148
+  c3.939,1.434,7.062,3.625,9.373,6.568L238.162,42.912z M153.559,72.816l8.533-2.576l1.676,5.156l-8.498,2.897l5.275,7.479
+  l-4.447,3.226l-5.553-7.349l-5.408,7.154l-4.318-3.289l5.275-7.223l-8.564-3.09l1.678-5.16l8.6,2.771v-8.896h5.754v8.897H153.559z
+   M124.086,45.836c-1.473-3.301-3.52-6.088-6.148-8.357c-2.625-2.268-5.711-4-9.252-5.193c-3.543-1.193-7.383-1.791-11.521-1.791
+  c-1.512,0-3.203,0.082-5.074,0.238c-1.871,0.162-3.482,0.439-4.834,0.838l0.834-18.268h34.503V0.41H74.481l-1.432,46.201
+  c1.271-0.635,2.725-1.232,4.357-1.791c1.631-0.555,3.301-1.053,5.014-1.49c1.711-0.438,3.463-0.775,5.254-1.016
+  c1.791-0.238,3.48-0.357,5.074-0.357c2.307,0,4.576,0.258,6.805,0.775c2.227,0.518,4.238,1.434,6.029,2.746s3.242,3.045,4.357,5.193
+  c1.113,2.148,1.672,4.855,1.672,8.119c0,2.547-0.418,4.836-1.254,6.865c-0.836,2.026-1.971,3.721-3.402,5.071
+  c-1.434,1.355-3.104,2.39-5.016,3.104c-1.91,0.719-3.939,1.076-6.088,1.076c-3.82,0-7.125-1.017-9.91-3.046
+  c-2.787-2.028-4.775-4.715-5.969-8.059l-0.16,0.059l-10.367,9.716c2.096,3.42,4.799,6.28,8.139,8.553
+  c4.854,3.302,10.824,4.955,17.91,4.955c4.217,0,8.197-0.678,11.938-2.028c3.741-1.352,7.004-3.304,9.791-5.853
+  c2.786-2.545,4.994-5.67,6.627-9.371c1.629-3.701,2.445-7.897,2.445-12.597C126.295,52.939,125.559,49.141,124.086,45.836z
+   M131.07,6.842h2.521c0.244,0,0.484,0.029,0.723,0.086c0.236,0.059,0.447,0.152,0.635,0.283c0.186,0.131,0.336,0.301,0.453,0.508
+  c0.115,0.207,0.172,0.457,0.172,0.749c0,0.365-0.104,0.667-0.311,0.904c-0.207,0.237-0.479,0.407-0.812,0.511v0.02
+  c0.408,0.055,0.742,0.213,1.006,0.475c0.262,0.262,0.393,0.611,0.393,1.051c0,0.354-0.07,0.65-0.209,0.891
+  c-0.143,0.24-0.324,0.434-0.555,0.58c-0.229,0.146-0.488,0.251-0.785,0.314c-0.295,0.064-0.596,0.096-0.898,0.096h-2.33V6.842
+  H131.07z M132.221,9.473h1.023c0.383,0,0.676-0.076,0.877-0.229c0.201-0.153,0.301-0.369,0.301-0.648c0-0.293-0.104-0.5-0.311-0.621
+  c-0.207-0.122-0.529-0.184-0.969-0.184h-0.924v1.682H132.221z M132.221,12.341h1.031c0.146,0,0.307-0.011,0.477-0.032
+  s0.328-0.064,0.471-0.133c0.143-0.066,0.262-0.164,0.355-0.292c0.096-0.128,0.143-0.298,0.143-0.511
+  c0-0.342-0.115-0.579-0.348-0.713c-0.23-0.135-0.582-0.201-1.051-0.201h-1.078V12.341z M136.936,6.842h4.283v1.004h-3.135v1.645
+  h2.969v0.969h-2.969v1.827h3.299v1.022h-4.447V6.842z M144.088,7.846h-1.982V6.842h5.117v1.004h-1.982v5.463h-1.152V7.846
+  L144.088,7.846z M149.449,6.842h0.996l2.787,6.467h-1.316l-0.602-1.479h-2.807l-0.584,1.479h-1.289L149.449,6.842z M150.912,10.843
+  l-0.996-2.631l-1.014,2.631H150.912z"/>
+</svg>

web/p5_export/p5-widget/static/index.css

@@ -0,0 +1,17 @@
+html {
+  min-width: 768px;
+}
+
+body {
+  font-family: Georgia, serif;
+  max-width: 740px;
+  margin: 0 auto;
+}
+
+h1, h2, h3 {
+  font-weight: normal;
+}
+
+a {
+  color: inherit;
+}

web/p5_export/p5-widget/static/vendor/codemirror.css

@@ -0,0 +1,338 @@
+/* BASICS */
+
+.CodeMirror {
+  /* Set height, width, borders, and global font properties here */
+  font-family: monospace;
+  height: 300px;
+  color: black;
+}
+
+/* PADDING */
+
+.CodeMirror-lines {
+  padding: 4px 0; /* Vertical padding around content */
+}
+.CodeMirror pre {
+  padding: 0 4px; /* Horizontal padding of content */
+}
+
+.CodeMirror-scrollbar-filler, .CodeMirror-gutter-filler {
+  background-color: white; /* The little square between H and V scrollbars */
+}
+
+/* GUTTER */
+
+.CodeMirror-gutters {
+  border-right: 1px solid #ddd;
+  background-color: #f7f7f7;
+  white-space: nowrap;
+}
+.CodeMirror-linenumbers {}
+.CodeMirror-linenumber {
+  padding: 0 3px 0 5px;
+  min-width: 20px;
+  text-align: right;
+  color: #999;
+  white-space: nowrap;
+}
+
+.CodeMirror-guttermarker { color: black; }
+.CodeMirror-guttermarker-subtle { color: #999; }
+
+/* CURSOR */
+
+.CodeMirror-cursor {
+  border-left: 1px solid black;
+  border-right: none;
+  width: 0;
+}
+/* Shown when moving in bi-directional text */
+.CodeMirror div.CodeMirror-secondarycursor {
+  border-left: 1px solid silver;
+}
+.cm-fat-cursor .CodeMirror-cursor {
+  width: auto;
+  border: 0;
+  background: #7e7;
+}
+.cm-fat-cursor div.CodeMirror-cursors {
+  z-index: 1;
+}
+
+.cm-animate-fat-cursor {
+  width: auto;
+  border: 0;
+  -webkit-animation: blink 1.06s steps(1) infinite;
+  -moz-animation: blink 1.06s steps(1) infinite;
+  animation: blink 1.06s steps(1) infinite;
+  background-color: #7e7;
+}
+@-moz-keyframes blink {
+  0% {}
+  50% { background-color: transparent; }
+  100% {}
+}
+@-webkit-keyframes blink {
+  0% {}
+  50% { background-color: transparent; }
+  100% {}
+}
+@keyframes blink {
+  0% {}
+  50% { background-color: transparent; }
+  100% {}
+}
+
+/* Can style cursor different in overwrite (non-insert) mode */
+.CodeMirror-overwrite .CodeMirror-cursor {}
+
+.cm-tab { display: inline-block; text-decoration: inherit; }
+
+.CodeMirror-ruler {
+  border-left: 1px solid #ccc;
+  position: absolute;
+}
+
+/* DEFAULT THEME */
+
+.cm-s-default .cm-header {color: blue;}
+.cm-s-default .cm-quote {color: #090;}
+.cm-negative {color: #d44;}
+.cm-positive {color: #292;}
+.cm-header, .cm-strong {font-weight: bold;}
+.cm-em {font-style: italic;}
+.cm-link {text-decoration: underline;}
+.cm-strikethrough {text-decoration: line-through;}
+
+.cm-s-default .cm-keyword {color: #708;}
+.cm-s-default .cm-atom {color: #219;}
+.cm-s-default .cm-number {color: #164;}
+.cm-s-default .cm-def {color: #00f;}
+.cm-s-default .cm-variable,
+.cm-s-default .cm-punctuation,
+.cm-s-default .cm-property,
+.cm-s-default .cm-operator {}
+.cm-s-default .cm-variable-2 {color: #05a;}
+.cm-s-default .cm-variable-3 {color: #085;}
+.cm-s-default .cm-comment {color: #a50;}
+.cm-s-default .cm-string {color: #a11;}
+.cm-s-default .cm-string-2 {color: #f50;}
+.cm-s-default .cm-meta {color: #555;}
+.cm-s-default .cm-qualifier {color: #555;}
+.cm-s-default .cm-builtin {color: #30a;}
+.cm-s-default .cm-bracket {color: #997;}
+.cm-s-default .cm-tag {color: #170;}
+.cm-s-default .cm-attribute {color: #00c;}
+.cm-s-default .cm-hr {color: #999;}
+.cm-s-default .cm-link {color: #00c;}
+
+.cm-s-default .cm-error {color: #f00;}
+.cm-invalidchar {color: #f00;}
+
+.CodeMirror-composing { border-bottom: 2px solid; }
+
+/* Default styles for common addons */
+
+div.CodeMirror span.CodeMirror-matchingbracket {color: #0f0;}
+div.CodeMirror span.CodeMirror-nonmatchingbracket {color: #f22;}
+.CodeMirror-matchingtag { background: rgba(255, 150, 0, .3); }
+.CodeMirror-activeline-background {background: #e8f2ff;}
+
+/* STOP */
+
+/* The rest of this file contains styles related to the mechanics of
+   the editor. You probably shouldn't touch them. */
+
+.CodeMirror {
+  position: relative;
+  overflow: hidden;
+  background: white;
+}
+
+.CodeMirror-scroll {
+  overflow: scroll !important; /* Things will break if this is overridden */
+  /* 30px is the magic margin used to hide the element's real scrollbars */
+  /* See overflow: hidden in .CodeMirror */
+  margin-bottom: -30px; margin-right: -30px;
+  padding-bottom: 30px;
+  height: 100%;
+  outline: none; /* Prevent dragging from highlighting the element */
+  position: relative;
+}
+.CodeMirror-sizer {
+  position: relative;
+  border-right: 30px solid transparent;
+}
+
+/* The fake, visible scrollbars. Used to force redraw during scrolling
+   before actual scrolling happens, thus preventing shaking and
+   flickering artifacts. */
+.CodeMirror-vscrollbar, .CodeMirror-hscrollbar, .CodeMirror-scrollbar-filler, .CodeMirror-gutter-filler {
+  position: absolute;
+  z-index: 6;
+  display: none;
+}
+.CodeMirror-vscrollbar {
+  right: 0; top: 0;
+  overflow-x: hidden;
+  overflow-y: scroll;
+}
+.CodeMirror-hscrollbar {
+  bottom: 0; left: 0;
+  overflow-y: hidden;
+  overflow-x: scroll;
+}
+.CodeMirror-scrollbar-filler {
+  right: 0; bottom: 0;
+}
+.CodeMirror-gutter-filler {
+  left: 0; bottom: 0;
+}
+
+.CodeMirror-gutters {
+  position: absolute; left: 0; top: 0;
+  min-height: 100%;
+  z-index: 3;
+}
+.CodeMirror-gutter {
+  white-space: normal;
+  height: 100%;
+  display: inline-block;
+  vertical-align: top;
+  margin-bottom: -30px;
+  /* Hack to make IE7 behave */
+  *zoom:1;
+  *display:inline;
+}
+.CodeMirror-gutter-wrapper {
+  position: absolute;
+  z-index: 4;
+  background: none !important;
+  border: none !important;
+}
+.CodeMirror-gutter-background {
+  position: absolute;
+  top: 0; bottom: 0;
+  z-index: 4;
+}
+.CodeMirror-gutter-elt {
+  position: absolute;
+  cursor: default;
+  z-index: 4;
+}
+.CodeMirror-gutter-wrapper {
+  -webkit-user-select: none;
+  -moz-user-select: none;
+  user-select: none;
+}
+
+.CodeMirror-lines {
+  cursor: text;
+  min-height: 1px; /* prevents collapsing before first draw */
+}
+.CodeMirror pre {
+  /* Reset some styles that the rest of the page might have set */
+  -moz-border-radius: 0; -webkit-border-radius: 0; border-radius: 0;
+  border-width: 0;
+  background: transparent;
+  font-family: inherit;
+  font-size: inherit;
+  margin: 0;
+  white-space: pre;
+  word-wrap: normal;
+  line-height: inherit;
+  color: inherit;
+  z-index: 2;
+  position: relative;
+  overflow: visible;
+  -webkit-tap-highlight-color: transparent;
+  -webkit-font-variant-ligatures: none;
+  font-variant-ligatures: none;
+}
+.CodeMirror-wrap pre {
+  word-wrap: break-word;
+  white-space: pre-wrap;
+  word-break: normal;
+}
+
+.CodeMirror-linebackground {
+  position: absolute;
+  left: 0; right: 0; top: 0; bottom: 0;
+  z-index: 0;
+}
+
+.CodeMirror-linewidget {
+  position: relative;
+  z-index: 2;
+  overflow: auto;
+}
+
+.CodeMirror-widget {}
+
+.CodeMirror-code {
+  outline: none;
+}
+
+/* Force content-box sizing for the elements where we expect it */
+.CodeMirror-scroll,
+.CodeMirror-sizer,
+.CodeMirror-gutter,
+.CodeMirror-gutters,
+.CodeMirror-linenumber {
+  -moz-box-sizing: content-box;
+  box-sizing: content-box;
+}
+
+.CodeMirror-measure {
+  position: absolute;
+  width: 100%;
+  height: 0;
+  overflow: hidden;
+  visibility: hidden;
+}
+
+.CodeMirror-cursor { position: absolute; }
+.CodeMirror-measure pre { position: static; }
+
+div.CodeMirror-cursors {
+  visibility: hidden;
+  position: relative;
+  z-index: 3;
+}
+div.CodeMirror-dragcursors {
+  visibility: visible;
+}
+
+.CodeMirror-focused div.CodeMirror-cursors {
+  visibility: visible;
+}
+
+.CodeMirror-selected { background: #d9d9d9; }
+.CodeMirror-focused .CodeMirror-selected { background: #d7d4f0; }
+.CodeMirror-crosshair { cursor: crosshair; }
+.CodeMirror-line::selection, .CodeMirror-line > span::selection, .CodeMirror-line > span > span::selection { background: #d7d4f0; }
+.CodeMirror-line::-moz-selection, .CodeMirror-line > span::-moz-selection, .CodeMirror-line > span > span::-moz-selection { background: #d7d4f0; }
+
+.cm-searching {
+  background: #ffa;
+  background: rgba(255, 255, 0, .4);
+}
+
+/* IE7 hack to prevent it from returning funny offsetTops on the spans */
+.CodeMirror span { *vertical-align: text-bottom; }
+
+/* Used to force a border model for a node */
+.cm-force-border { padding-right: .1px; }
+
+@media print {
+  /* Hide the cursor when printing */
+  .CodeMirror div.CodeMirror-cursors {
+    visibility: hidden;
+  }
+}
+
+/* See issue #2901 */
+.cm-tab-wrap-hack:after { content: ''; }
+
+/* Help users use markselection to safely style text background */
+span.CodeMirror-selectedtext { background: none; }

web/p5_export/p5-widget/static/vendor/prism.css

@@ -0,0 +1,123 @@
+/* http://prismjs.com/download.html?themes=prism-okaidia&languages=markup+css+clike+javascript */
+/**
+ * okaidia theme for JavaScript, CSS and HTML
+ * Loosely based on Monokai textmate theme by http://www.monokai.nl/
+ * @author ocodia
+ */
+
+code[class*="language-"],
+pre[class*="language-"] {
+	color: #f8f8f2;
+	background: none;
+	text-shadow: 0 1px rgba(0, 0, 0, 0.3);
+	font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace;
+	text-align: left;
+	white-space: pre;
+	word-spacing: normal;
+	word-break: normal;
+	word-wrap: normal;
+	line-height: 1.5;
+
+	-moz-tab-size: 4;
+	-o-tab-size: 4;
+	tab-size: 4;
+
+	-webkit-hyphens: none;
+	-moz-hyphens: none;
+	-ms-hyphens: none;
+	hyphens: none;
+}
+
+/* Code blocks */
+pre[class*="language-"] {
+	padding: 1em;
+	margin: .5em 0;
+	overflow: auto;
+	border-radius: 0.3em;
+}
+
+:not(pre) > code[class*="language-"],
+pre[class*="language-"] {
+	background: #272822;
+}
+
+/* Inline code */
+:not(pre) > code[class*="language-"] {
+	padding: .1em;
+	border-radius: .3em;
+	white-space: normal;
+}
+
+.token.comment,
+.token.prolog,
+.token.doctype,
+.token.cdata {
+	color: slategray;
+}
+
+.token.punctuation {
+	color: #f8f8f2;
+}
+
+.namespace {
+	opacity: .7;
+}
+
+.token.property,
+.token.tag,
+.token.constant,
+.token.symbol,
+.token.deleted {
+	color: #f92672;
+}
+
+.token.boolean,
+.token.number {
+	color: #ae81ff;
+}
+
+.token.selector,
+.token.attr-name,
+.token.string,
+.token.char,
+.token.builtin,
+.token.inserted {
+	color: #a6e22e;
+}
+
+.token.operator,
+.token.entity,
+.token.url,
+.language-css .token.string,
+.style .token.string,
+.token.variable {
+	color: #f8f8f2;
+}
+
+.token.atrule,
+.token.attr-value,
+.token.function {
+	color: #e6db74;
+}
+
+.token.keyword {
+	color: #66d9ef;
+}
+
+.token.regex,
+.token.important {
+	color: #fd971f;
+}
+
+.token.important,
+.token.bold {
+	font-weight: bold;
+}
+.token.italic {
+	font-style: italic;
+}
+
+.token.entity {
+	cursor: help;
+}
+

web/p5_export/p5-widget/static/vendor/prism.js

@@ -0,0 +1,668 @@
+/* http://prismjs.com/download.html?themes=prism&languages=markup+css+clike+javascript */
+var _self = (typeof window !== 'undefined')
+	? window   // if in browser
+	: (
+		(typeof WorkerGlobalScope !== 'undefined' && self instanceof WorkerGlobalScope)
+		? self // if in worker
+		: {}   // if in node js
+	);
+
+/**
+ * Prism: Lightweight, robust, elegant syntax highlighting
+ * MIT license http://www.opensource.org/licenses/mit-license.php/
+ * @author Lea Verou http://lea.verou.me
+ */
+
+var Prism = (function(){
+
+// Private helper vars
+var lang = /\blang(?:uage)?-(\w+)\b/i;
+var uniqueId = 0;
+
+var _ = _self.Prism = {
+	util: {
+		encode: function (tokens) {
+			if (tokens instanceof Token) {
+				return new Token(tokens.type, _.util.encode(tokens.content), tokens.alias);
+			} else if (_.util.type(tokens) === 'Array') {
+				return tokens.map(_.util.encode);
+			} else {
+				return tokens.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/\u00a0/g, ' ');
+			}
+		},
+
+		type: function (o) {
+			return Object.prototype.toString.call(o).match(/\[object (\w+)\]/)[1];
+		},
+
+		objId: function (obj) {
+			if (!obj['__id']) {
+				Object.defineProperty(obj, '__id', { value: ++uniqueId });
+			}
+			return obj['__id'];
+		},
+
+		// Deep clone a language definition (e.g. to extend it)
+		clone: function (o) {
+			var type = _.util.type(o);
+
+			switch (type) {
+				case 'Object':
+					var clone = {};
+
+					for (var key in o) {
+						if (o.hasOwnProperty(key)) {
+							clone[key] = _.util.clone(o[key]);
+						}
+					}
+
+					return clone;
+
+				case 'Array':
+					// Check for existence for IE8
+					return o.map && o.map(function(v) { return _.util.clone(v); });
+			}
+
+			return o;
+		}
+	},
+
+	languages: {
+		extend: function (id, redef) {
+			var lang = _.util.clone(_.languages[id]);
+
+			for (var key in redef) {
+				lang[key] = redef[key];
+			}
+
+			return lang;
+		},
+
+		/**
+		 * Insert a token before another token in a language literal
+		 * As this needs to recreate the object (we cannot actually insert before keys in object literals),
+		 * we cannot just provide an object, we need anobject and a key.
+		 * @param inside The key (or language id) of the parent
+		 * @param before The key to insert before. If not provided, the function appends instead.
+		 * @param insert Object with the key/value pairs to insert
+		 * @param root The object that contains `inside`. If equal to Prism.languages, it can be omitted.
+		 */
+		insertBefore: function (inside, before, insert, root) {
+			root = root || _.languages;
+			var grammar = root[inside];
+
+			if (arguments.length == 2) {
+				insert = arguments[1];
+
+				for (var newToken in insert) {
+					if (insert.hasOwnProperty(newToken)) {
+						grammar[newToken] = insert[newToken];
+					}
+				}
+
+				return grammar;
+			}
+
+			var ret = {};
+
+			for (var token in grammar) {
+
+				if (grammar.hasOwnProperty(token)) {
+
+					if (token == before) {
+
+						for (var newToken in insert) {
+
+							if (insert.hasOwnProperty(newToken)) {
+								ret[newToken] = insert[newToken];
+							}
+						}
+					}
+
+					ret[token] = grammar[token];
+				}
+			}
+
+			// Update references in other language definitions
+			_.languages.DFS(_.languages, function(key, value) {
+				if (value === root[inside] && key != inside) {
+					this[key] = ret;
+				}
+			});
+
+			return root[inside] = ret;
+		},
+
+		// Traverse a language definition with Depth First Search
+		DFS: function(o, callback, type, visited) {
+			visited = visited || {};
+			for (var i in o) {
+				if (o.hasOwnProperty(i)) {
+					callback.call(o, i, o[i], type || i);
+
+					if (_.util.type(o[i]) === 'Object' && !visited[_.util.objId(o[i])]) {
+						visited[_.util.objId(o[i])] = true;
+						_.languages.DFS(o[i], callback, null, visited);
+					}
+					else if (_.util.type(o[i]) === 'Array' && !visited[_.util.objId(o[i])]) {
+						visited[_.util.objId(o[i])] = true;
+						_.languages.DFS(o[i], callback, i, visited);
+					}
+				}
+			}
+		}
+	},
+	plugins: {},
+
+	highlightAll: function(async, callback) {
+		var env = {
+			callback: callback,
+			selector: 'code[class*="language-"], [class*="language-"] code, code[class*="lang-"], [class*="lang-"] code'
+		};
+
+		_.hooks.run("before-highlightall", env);
+
+		var elements = env.elements || document.querySelectorAll(env.selector);
+
+		for (var i=0, element; element = elements[i++];) {
+			_.highlightElement(element, async === true, env.callback);
+		}
+	},
+
+	highlightElement: function(element, async, callback) {
+		// Find language
+		var language, grammar, parent = element;
+
+		while (parent && !lang.test(parent.className)) {
+			parent = parent.parentNode;
+		}
+
+		if (parent) {
+			language = (parent.className.match(lang) || [,''])[1];
+			grammar = _.languages[language];
+		}
+
+		// Set language on the element, if not present
+		element.className = element.className.replace(lang, '').replace(/\s+/g, ' ') + ' language-' + language;
+
+		// Set language on the parent, for styling
+		parent = element.parentNode;
+
+		if (/pre/i.test(parent.nodeName)) {
+			parent.className = parent.className.replace(lang, '').replace(/\s+/g, ' ') + ' language-' + language;
+		}
+
+		var code = element.textContent;
+
+		var env = {
+			element: element,
+			language: language,
+			grammar: grammar,
+			code: code
+		};
+
+		if (!code || !grammar) {
+			_.hooks.run('complete', env);
+			return;
+		}
+
+		_.hooks.run('before-highlight', env);
+
+		if (async && _self.Worker) {
+			var worker = new Worker(_.filename);
+
+			worker.onmessage = function(evt) {
+				env.highlightedCode = evt.data;
+
+				_.hooks.run('before-insert', env);
+
+				env.element.innerHTML = env.highlightedCode;
+
+				callback && callback.call(env.element);
+				_.hooks.run('after-highlight', env);
+				_.hooks.run('complete', env);
+			};
+
+			worker.postMessage(JSON.stringify({
+				language: env.language,
+				code: env.code,
+				immediateClose: true
+			}));
+		}
+		else {
+			env.highlightedCode = _.highlight(env.code, env.grammar, env.language);
+
+			_.hooks.run('before-insert', env);
+
+			env.element.innerHTML = env.highlightedCode;
+
+			callback && callback.call(element);
+
+			_.hooks.run('after-highlight', env);
+			_.hooks.run('complete', env);
+		}
+	},
+
+	highlight: function (text, grammar, language) {
+		var tokens = _.tokenize(text, grammar);
+		return Token.stringify(_.util.encode(tokens), language);
+	},
+
+	tokenize: function(text, grammar, language) {
+		var Token = _.Token;
+
+		var strarr = [text];
+
+		var rest = grammar.rest;
+
+		if (rest) {
+			for (var token in rest) {
+				grammar[token] = rest[token];
+			}
+
+			delete grammar.rest;
+		}
+
+		tokenloop: for (var token in grammar) {
+			if(!grammar.hasOwnProperty(token) || !grammar[token]) {
+				continue;
+			}
+
+			var patterns = grammar[token];
+			patterns = (_.util.type(patterns) === "Array") ? patterns : [patterns];
+
+			for (var j = 0; j < patterns.length; ++j) {
+				var pattern = patterns[j],
+					inside = pattern.inside,
+					lookbehind = !!pattern.lookbehind,
+					greedy = !!pattern.greedy,
+					lookbehindLength = 0,
+					alias = pattern.alias;
+
+				pattern = pattern.pattern || pattern;
+
+				for (var i=0; i<strarr.length; i++) { // Don’t cache length as it changes during the loop
+
+					var str = strarr[i];
+
+					if (strarr.length > text.length) {
+						// Something went terribly wrong, ABORT, ABORT!
+						break tokenloop;
+					}
+
+					if (str instanceof Token) {
+						continue;
+					}
+
+					pattern.lastIndex = 0;
+
+					var match = pattern.exec(str),
+					    delNum = 1;
+
+					// Greedy patterns can override/remove up to two previously matched tokens
+					if (!match && greedy && i != strarr.length - 1) {
+						// Reconstruct the original text using the next two tokens
+						var nextToken = strarr[i + 1].matchedStr || strarr[i + 1],
+						    combStr = str + nextToken;
+
+						if (i < strarr.length - 2) {
+							combStr += strarr[i + 2].matchedStr || strarr[i + 2];
+						}
+
+						// Try the pattern again on the reconstructed text
+						pattern.lastIndex = 0;
+						match = pattern.exec(combStr);
+						if (!match) {
+							continue;
+						}
+
+						var from = match.index + (lookbehind ? match[1].length : 0);
+						// To be a valid candidate, the new match has to start inside of str
+						if (from >= str.length) {
+							continue;
+						}
+						var to = match.index + match[0].length,
+						    len = str.length + nextToken.length;
+
+						// Number of tokens to delete and replace with the new match
+						delNum = 3;
+
+						if (to <= len) {
+							if (strarr[i + 1].greedy) {
+								continue;
+							}
+							delNum = 2;
+							combStr = combStr.slice(0, len);
+						}
+						str = combStr;
+					}
+
+					if (!match) {
+						continue;
+					}
+
+					if(lookbehind) {
+						lookbehindLength = match[1].length;
+					}
+
+					var from = match.index + lookbehindLength,
+					    match = match[0].slice(lookbehindLength),
+					    to = from + match.length,
+					    before = str.slice(0, from),
+					    after = str.slice(to);
+
+					var args = [i, delNum];
+
+					if (before) {
+						args.push(before);
+					}
+
+					var wrapped = new Token(token, inside? _.tokenize(match, inside) : match, alias, match, greedy);
+
+					args.push(wrapped);
+
+					if (after) {
+						args.push(after);
+					}
+
+					Array.prototype.splice.apply(strarr, args);
+				}
+			}
+		}
+
+		return strarr;
+	},
+
+	hooks: {
+		all: {},
+
+		add: function (name, callback) {
+			var hooks = _.hooks.all;
+
+			hooks[name] = hooks[name] || [];
+
+			hooks[name].push(callback);
+		},
+
+		run: function (name, env) {
+			var callbacks = _.hooks.all[name];
+
+			if (!callbacks || !callbacks.length) {
+				return;
+			}
+
+			for (var i=0, callback; callback = callbacks[i++];) {
+				callback(env);
+			}
+		}
+	}
+};
+
+var Token = _.Token = function(type, content, alias, matchedStr, greedy) {
+	this.type = type;
+	this.content = content;
+	this.alias = alias;
+	// Copy of the full string this token was created from
+	this.matchedStr = matchedStr || null;
+	this.greedy = !!greedy;
+};
+
+Token.stringify = function(o, language, parent) {
+	if (typeof o == 'string') {
+		return o;
+	}
+
+	if (_.util.type(o) === 'Array') {
+		return o.map(function(element) {
+			return Token.stringify(element, language, o);
+		}).join('');
+	}
+
+	var env = {
+		type: o.type,
+		content: Token.stringify(o.content, language, parent),
+		tag: 'span',
+		classes: ['token', o.type],
+		attributes: {},
+		language: language,
+		parent: parent
+	};
+
+	if (env.type == 'comment') {
+		env.attributes['spellcheck'] = 'true';
+	}
+
+	if (o.alias) {
+		var aliases = _.util.type(o.alias) === 'Array' ? o.alias : [o.alias];
+		Array.prototype.push.apply(env.classes, aliases);
+	}
+
+	_.hooks.run('wrap', env);
+
+	var attributes = '';
+
+	for (var name in env.attributes) {
+		attributes += (attributes ? ' ' : '') + name + '="' + (env.attributes[name] || '') + '"';
+	}
+
+	return '<' + env.tag + ' class="' + env.classes.join(' ') + '" ' + attributes + '>' + env.content + '</' + env.tag + '>';
+
+};
+
+if (!_self.document) {
+	if (!_self.addEventListener) {
+		// in Node.js
+		return _self.Prism;
+	}
+ 	// In worker
+	_self.addEventListener('message', function(evt) {
+		var message = JSON.parse(evt.data),
+		    lang = message.language,
+		    code = message.code,
+		    immediateClose = message.immediateClose;
+
+		_self.postMessage(_.highlight(code, _.languages[lang], lang));
+		if (immediateClose) {
+			_self.close();
+		}
+	}, false);
+
+	return _self.Prism;
+}
+
+//Get current script and highlight
+var script = document.currentScript || [].slice.call(document.getElementsByTagName("script")).pop();
+
+if (script) {
+	_.filename = script.src;
+
+	if (document.addEventListener && !script.hasAttribute('data-manual')) {
+		document.addEventListener('DOMContentLoaded', _.highlightAll);
+	}
+}
+
+return _self.Prism;
+
+})();
+
+if (typeof module !== 'undefined' && module.exports) {
+	module.exports = Prism;
+}
+
+// hack for components to work correctly in node.js
+if (typeof global !== 'undefined') {
+	global.Prism = Prism;
+}
+;
+Prism.languages.markup = {
+	'comment': /<!--[\w\W]*?-->/,
+	'prolog': /<\?[\w\W]+?\?>/,
+	'doctype': /<!DOCTYPE[\w\W]+?>/,
+	'cdata': /<!\[CDATA\[[\w\W]*?]]>/i,
+	'tag': {
+		pattern: /<\/?(?!\d)[^\s>\/=.$<]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\\1|\\?(?!\1)[\w\W])*\1|[^\s'">=]+))?)*\s*\/?>/i,
+		inside: {
+			'tag': {
+				pattern: /^<\/?[^\s>\/]+/i,
+				inside: {
+					'punctuation': /^<\/?/,
+					'namespace': /^[^\s>\/:]+:/
+				}
+			},
+			'attr-value': {
+				pattern: /=(?:('|")[\w\W]*?(\1)|[^\s>]+)/i,
+				inside: {
+					'punctuation': /[=>"']/
+				}
+			},
+			'punctuation': /\/?>/,
+			'attr-name': {
+				pattern: /[^\s>\/]+/,
+				inside: {
+					'namespace': /^[^\s>\/:]+:/
+				}
+			}
+
+		}
+	},
+	'entity': /&#?[\da-z]{1,8};/i
+};
+
+// Plugin to make entity title show the real entity, idea by Roman Komarov
+Prism.hooks.add('wrap', function(env) {
+
+	if (env.type === 'entity') {
+		env.attributes['title'] = env.content.replace(/&amp;/, '&');
+	}
+});
+
+Prism.languages.xml = Prism.languages.markup;
+Prism.languages.html = Prism.languages.markup;
+Prism.languages.mathml = Prism.languages.markup;
+Prism.languages.svg = Prism.languages.markup;
+
+Prism.languages.css = {
+	'comment': /\/\*[\w\W]*?\*\//,
+	'atrule': {
+		pattern: /@[\w-]+?.*?(;|(?=\s*\{))/i,
+		inside: {
+			'rule': /@[\w-]+/
+			// See rest below
+		}
+	},
+	'url': /url\((?:(["'])(\\(?:\r\n|[\w\W])|(?!\1)[^\\\r\n])*\1|.*?)\)/i,
+	'selector': /[^\{\}\s][^\{\};]*?(?=\s*\{)/,
+	'string': /("|')(\\(?:\r\n|[\w\W])|(?!\1)[^\\\r\n])*\1/,
+	'property': /(\b|\B)[\w-]+(?=\s*:)/i,
+	'important': /\B!important\b/i,
+	'function': /[-a-z0-9]+(?=\()/i,
+	'punctuation': /[(){};:]/
+};
+
+Prism.languages.css['atrule'].inside.rest = Prism.util.clone(Prism.languages.css);
+
+if (Prism.languages.markup) {
+	Prism.languages.insertBefore('markup', 'tag', {
+		'style': {
+			pattern: /(<style[\w\W]*?>)[\w\W]*?(?=<\/style>)/i,
+			lookbehind: true,
+			inside: Prism.languages.css,
+			alias: 'language-css'
+		}
+	});
+	
+	Prism.languages.insertBefore('inside', 'attr-value', {
+		'style-attr': {
+			pattern: /\s*style=("|').*?\1/i,
+			inside: {
+				'attr-name': {
+					pattern: /^\s*style/i,
+					inside: Prism.languages.markup.tag.inside
+				},
+				'punctuation': /^\s*=\s*['"]|['"]\s*$/,
+				'attr-value': {
+					pattern: /.+/i,
+					inside: Prism.languages.css
+				}
+			},
+			alias: 'language-css'
+		}
+	}, Prism.languages.markup.tag);
+};
+Prism.languages.clike = {
+	'comment': [
+		{
+			pattern: /(^|[^\\])\/\*[\w\W]*?\*\//,
+			lookbehind: true
+		},
+		{
+			pattern: /(^|[^\\:])\/\/.*/,
+			lookbehind: true
+		}
+	],
+	'string': {
+		pattern: /(["'])(\\(?:\r\n|[\s\S])|(?!\1)[^\\\r\n])*\1/,
+		greedy: true
+	},
+	'class-name': {
+		pattern: /((?:\b(?:class|interface|extends|implements|trait|instanceof|new)\s+)|(?:catch\s+\())[a-z0-9_\.\\]+/i,
+		lookbehind: true,
+		inside: {
+			punctuation: /(\.|\\)/
+		}
+	},
+	'keyword': /\b(if|else|while|do|for|return|in|instanceof|function|new|try|throw|catch|finally|null|break|continue)\b/,
+	'boolean': /\b(true|false)\b/,
+	'function': /[a-z0-9_]+(?=\()/i,
+	'number': /\b-?(?:0x[\da-f]+|\d*\.?\d+(?:e[+-]?\d+)?)\b/i,
+	'operator': /--?|\+\+?|!=?=?|<=?|>=?|==?=?|&&?|\|\|?|\?|\*|\/|~|\^|%/,
+	'punctuation': /[{}[\];(),.:]/
+};
+
+Prism.languages.javascript = Prism.languages.extend('clike', {
+	'keyword': /\b(as|async|await|break|case|catch|class|const|continue|debugger|default|delete|do|else|enum|export|extends|finally|for|from|function|get|if|implements|import|in|instanceof|interface|let|new|null|of|package|private|protected|public|return|set|static|super|switch|this|throw|try|typeof|var|void|while|with|yield)\b/,
+	'number': /\b-?(0x[\dA-Fa-f]+|0b[01]+|0o[0-7]+|\d*\.?\d+([Ee][+-]?\d+)?|NaN|Infinity)\b/,
+	// Allow for all non-ASCII characters (See http://stackoverflow.com/a/2008444)
+	'function': /[_$a-zA-Z\xA0-\uFFFF][_$a-zA-Z0-9\xA0-\uFFFF]*(?=\()/i
+});
+
+Prism.languages.insertBefore('javascript', 'keyword', {
+	'regex': {
+		pattern: /(^|[^/])\/(?!\/)(\[.+?]|\\.|[^/\\\r\n])+\/[gimyu]{0,5}(?=\s*($|[\r\n,.;})]))/,
+		lookbehind: true,
+		greedy: true
+	}
+});
+
+Prism.languages.insertBefore('javascript', 'class-name', {
+	'template-string': {
+		pattern: /`(?:\\\\|\\?[^\\])*?`/,
+		greedy: true,
+		inside: {
+			'interpolation': {
+				pattern: /\$\{[^}]+\}/,
+				inside: {
+					'interpolation-punctuation': {
+						pattern: /^\$\{|\}$/,
+						alias: 'punctuation'
+					},
+					rest: Prism.languages.javascript
+				}
+			},
+			'string': /[\s\S]+/
+		}
+	}
+});
+
+if (Prism.languages.markup) {
+	Prism.languages.insertBefore('markup', 'tag', {
+		'script': {
+			pattern: /(<script[\w\W]*?>)[\w\W]*?(?=<\/script>)/i,
+			lookbehind: true,
+			inside: Prism.languages.javascript,
+			alias: 'language-javascript'
+		}
+	});
+}
+
+Prism.languages.js = Prism.languages.javascript;

web/p5_export/p5.html

@@ -0,0 +1,105 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>Document</title>
+
+
+  <style>
+    body {
+      height: 600px;
+      padding: 24px;
+    }
+
+    iframe {
+      min-height: 600px !important
+    }
+
+    /* 自定义滚动条样式 */
+    ::-webkit-scrollbar {
+      width: 8px;
+      /* 滚动条宽度 */
+    }
+
+    ::-webkit-scrollbar-track {
+      background: #f1f1f1;
+      /* 滚动条轨道颜色 */
+    }
+
+    ::-webkit-scrollbar-thumb {
+      background: #888;
+      /* 滚动条滑块颜色 */
+    }
+
+    ::-webkit-scrollbar-thumb:hover {
+      background: #555;
+      /* 滚动条滑块悬停颜色 */
+    }
+  </style>
+  <!-- <script src="./p5.js"></script> -->
+  <!-- <script src="./sketch.js"></script> -->
+  <!-- <script src="./src/CCapture.js"></script> -->
+  <!-- <script src="./src/gif.js"></script>
+   <script src="./src/gif.worker.js"></script> -->
+  <!-- <script src="./src/download.js"></script> -->
+  <!-- <script src="./src/webm-writer-0.2.0.js"></script> -->
+
+</head>
+
+<body>
+
+  <script>
+
+    function getIdFromUrl(url) {
+      const urlParams = new URLSearchParams(new URL(url).search);
+      return urlParams.get('id');
+    }
+
+
+
+    // 监听来自iframe的消息
+    window.addEventListener('message', (event) => {
+      const data = event.data;
+      const nodeId = getIdFromUrl(window.location.href);
+
+      if (data.from === 'p5.widget' && data.status === 'save') {
+        const frames = data.frames;
+        // 示例用法
+        // const url = 'https://example.com/page?id=12345';
+        window.parent.postMessage({
+          frames,
+          from: 'p5.widget',
+          status: 'save',
+          nodeId
+        }, '*');
+        window.location.reload()
+      }
+      // console.log(data)
+      // if (data.from === 'p5.widget' && data.status === 'capture') {
+      //   window.parent.postMessage({
+      //     from: 'p5.widget',
+      //     status: 'capture',
+      //     frameCount: data.frameCount,
+      //     maxCount: data.maxCount,
+      //     nodeId
+      //   }, '*');
+      // }
+    });
+
+
+  </script>
+  <script type="text/p5" data-height="500" data-preview-width="300">
+function setup() {
+createCanvas(100, 100);
+}
+    
+function draw() {
+background(255, 0, 200);
+}
+    </script>
+  <script src="./p5-widget/p5-widget.js"></script>
+</body>
+
+</html>

web/p5_export/sketch.js

@@ -0,0 +1,70 @@
+function setup () {
+  createCanvas(400, 400, WEBGL)
+  angleMode(DEGREES)
+}
+
+function draw () {
+  if (frameCount === 1) {
+    capturer.start()
+  }
+
+  background(30)
+  noStroke()
+  translate(0, 0, sin(frameCount) * 400 - 800)
+
+  rotateX(frameCount)
+  rotateY(frameCount)
+  rotateZ(frameCount)
+
+  var w = 20
+
+  randomSeed(1)
+  for (var x = -width / 2; x <= width / 2; x += w) {
+    for (var y = -width / 2; y <= width / 2; y += w) {
+      for (var z = -width / 2; z <= width / 2; z += w) {
+        var r = random(255)
+        var g = random(255)
+        var b = random(255)
+
+        fill(r, g, b)
+
+        push()
+        translate(x, y, z)
+        box(w)
+        pop()
+      }
+    }
+  }
+  // console.log(frameRate());
+
+  if (frameCount < 60) {
+    capturer.capture(canvas)
+  } else if (frameCount === 60) {
+    capturer.save(function (blob) {
+        // console.log(blob)
+
+        // 示例用法
+        // const blob = new Blob([/* 数据 */], { type: 'video/webm' });
+        blobToBase64(blob).then(base64String => {
+            console.log(base64String);
+         
+            const video = document.createElement('video');
+            video.controls = true; // 显示视频控件（播放、暂停等）
+            video.src = base64String; // 设置视频的 src 属性为 Base64 数据 URL
+            video.width = 640; // 设置视频宽度
+            video.height = 360; // 设置视频高度
+
+            // 将 video 元素添加到页面中
+            document.body.appendChild(video)
+
+            // 自动播放视频
+            video.play();
+
+        }).catch(error => {
+            console.error('转换失败:', error);
+        });
+
+    })
+    capturer.stop()
+  }
+}

workflow/LLM_base_workflow.json

@@ -0,0 +1,408 @@
+{
+  "last_node_id": 21,
+  "last_link_id": 16,
+  "nodes": [
+    {
+      "id": 10,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        489,
+        689
+      ],
+      "size": {
+        "0": 403.2580261230469,
+        "1": 309.2166442871094
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "api_key",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "api_key"
+          }
+        },
+        {
+          "name": "custom_model_name",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "custom_model_name"
+          }
+        },
+        {
+          "name": "custom_api_url",
+          "type": "STRING",
+          "link": 13,
+          "widget": {
+            "name": "custom_api_url"
+          },
+          "slot_index": 2
+        }
+      ],
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            12
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        "hi",
+        "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
+        "gpt-3.5-turbo",
+        447210757728856,
+        "randomize",
+        1,
+        "openai",
+        "",
+        "",
+        ""
+      ]
+    },
+    {
+      "id": 3,
+      "type": "ShowTextForGPT",
+      "pos": [
+        982,
+        686
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 12,
+          "widget": {
+            "name": "text"
+          }
+        },
+        {
+          "name": "output_dir",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "output_dir"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowTextForGPT"
+      },
+      "widgets_values": [
+        "",
+        "",
+        " Hi there! What can I help you with?"
+      ]
+    },
+    {
+      "id": 11,
+      "type": "SiliconflowLLM",
+      "pos": [
+        489,
+        318
+      ],
+      "size": {
+        "0": 395.197998046875,
+        "1": 262
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "api_key",
+          "type": "STRING",
+          "link": 16,
+          "widget": {
+            "name": "api_key"
+          }
+        },
+        {
+          "name": "custom_model_name",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "custom_model_name"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            15
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SiliconflowLLM"
+      },
+      "widgets_values": [
+        "",
+        "",
+        "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
+        "Qwen/Qwen2-7B-Instruct",
+        593422808835285,
+        "randomize",
+        1,
+        ""
+      ]
+    },
+    {
+      "id": 17,
+      "type": "ShowTextForGPT",
+      "pos": [
+        975,
+        329
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 15,
+          "widget": {
+            "name": "text"
+          }
+        },
+        {
+          "name": "output_dir",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "output_dir"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowTextForGPT"
+      },
+      "widgets_values": [
+        "",
+        "",
+        "Hello! How can I assist you today?"
+      ]
+    },
+    {
+      "id": 18,
+      "type": "KeyInput",
+      "pos": [
+        46,
+        319
+      ],
+      "size": {
+        "0": 315,
+        "1": 70
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "key",
+          "type": "STRING",
+          "links": [
+            16
+          ],
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KeyInput"
+      },
+      "widgets_values": [
+        null,
+        null
+      ]
+    },
+    {
+      "id": 12,
+      "type": "TextInput_",
+      "pos": [
+        31,
+        871
+      ],
+      "size": [
+        407.9377612789413,
+        76
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": [
+            13
+          ],
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "TextInput_"
+      },
+      "widgets_values": [
+        "http://127.0.0.1:8000/v1"
+      ]
+    },
+    {
+      "id": 20,
+      "type": "Note",
+      "pos": [
+        35,
+        664
+      ],
+      "size": [
+        350.04604707424306,
+        116.54209784249178
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "api_key 填写对应平台的Key\ncustom model和api 根据需要自行填写\n\n如果不填写custome，则按照model和api_url选择的选项"
+      ],
+      "color": "#432",
+      "bgcolor": "#653"
+    },
+    {
+      "id": 21,
+      "type": "Note",
+      "pos": [
+        42,
+        438
+      ],
+      "size": {
+        "0": 350.0460510253906,
+        "1": 116.54209899902344
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "API key节点不会保存到workflow的json文件。\n\n：：会保存到appinfo导出的app.json里\n\n\n注册https://cloud.siliconflow.cn/account/ak 领取免费的API"
+      ],
+      "color": "#432",
+      "bgcolor": "#653"
+    }
+  ],
+  "links": [
+    [
+      12,
+      10,
+      0,
+      3,
+      0,
+      "STRING"
+    ],
+    [
+      13,
+      12,
+      0,
+      10,
+      2,
+      "STRING"
+    ],
+    [
+      15,
+      11,
+      0,
+      17,
+      0,
+      "STRING"
+    ],
+    [
+      16,
+      18,
+      0,
+      11,
+      0,
+      "STRING"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 0.9646149645000006,
+      "offset": [
+        170.81398913081276,
+        -128.0066534315481
+      ]
+    }
+  },
+  "version": 0.4
+}

workflow/text-to-audio-base-workflow.json

@@ -0,0 +1,415 @@
+{
+  "last_node_id": 20,
+  "last_link_id": 28,
+  "nodes": [
+    {
+      "id": 11,
+      "type": "EmptyLatentAudio",
+      "pos": [
+        576,
+        480
+      ],
+      "size": {
+        "0": 240,
+        "1": 58
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            12
+          ],
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptyLatentAudio"
+      },
+      "widgets_values": [
+        47.6
+      ]
+    },
+    {
+      "id": 7,
+      "type": "CLIPTextEncode",
+      "pos": [
+        384,
+        288
+      ],
+      "size": {
+        "0": 432,
+        "1": 144
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 26
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            6
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ],
+      "color": "#322",
+      "bgcolor": "#533"
+    },
+    {
+      "id": 3,
+      "type": "KSampler",
+      "pos": [
+        864,
+        96
+      ],
+      "size": {
+        "0": 315,
+        "1": 262
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 18
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 4
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 6
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 12,
+          "slot_index": 3
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            13
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        173246216057483,
+        "randomize",
+        50,
+        4.98,
+        "dpmpp_3m_sde_gpu",
+        "exponential",
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "CLIPTextEncode",
+      "pos": [
+        384,
+        96
+      ],
+      "size": {
+        "0": 432,
+        "1": 144
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            4
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "heaven church electronic dance music"
+      ],
+      "color": "#232",
+      "bgcolor": "#353"
+    },
+    {
+      "id": 12,
+      "type": "VAEDecodeAudio",
+      "pos": [
+        1200,
+        96
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 13
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 14,
+          "slot_index": 1
+        }
+      ],
+      "outputs": [
+        {
+          "name": "AUDIO",
+          "type": "AUDIO",
+          "links": [
+            28
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecodeAudio"
+      }
+    },
+    {
+      "id": 19,
+      "type": "AudioPlay",
+      "pos": [
+        1482,
+        96
+      ],
+      "size": [
+        210,
+        280
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "audio",
+          "type": "AUDIO",
+          "link": 28
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "AudioPlay"
+      },
+      "widgets_values": [
+        [
+          "/view?filename=._00001_.wav&type=temp&subfolder=&rand=0.5323617364976663",
+          null
+        ]
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CheckpointLoaderSimple",
+      "pos": [
+        -56,
+        304
+      ],
+      "size": [
+        390.12250902572447,
+        98
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            18
+          ],
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [],
+          "slot_index": 1
+        },
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            14
+          ],
+          "slot_index": 2
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CheckpointLoaderSimple"
+      },
+      "widgets_values": [
+        "stable_audio_open_1.0.safetensors"
+      ]
+    },
+    {
+      "id": 10,
+      "type": "CLIPLoader",
+      "pos": [
+        -18,
+        93
+      ],
+      "size": {
+        "0": 335.6534118652344,
+        "1": 82
+      },
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            25,
+            26
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader"
+      },
+      "widgets_values": [
+        "t5_base.safetensors",
+        "stable_audio"
+      ]
+    }
+  ],
+  "links": [
+    [
+      4,
+      6,
+      0,
+      3,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      6,
+      7,
+      0,
+      3,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      12,
+      11,
+      0,
+      3,
+      3,
+      "LATENT"
+    ],
+    [
+      13,
+      3,
+      0,
+      12,
+      0,
+      "LATENT"
+    ],
+    [
+      14,
+      4,
+      2,
+      12,
+      1,
+      "VAE"
+    ],
+    [
+      18,
+      4,
+      0,
+      3,
+      0,
+      "MODEL"
+    ],
+    [
+      25,
+      10,
+      0,
+      6,
+      0,
+      "CLIP"
+    ],
+    [
+      26,
+      10,
+      0,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      28,
+      12,
+      0,
+      19,
+      0,
+      "AUDIO"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 0.7972024500000005,
+      "offset": {
+        "0": 206.23990767458133,
+        "1": 402.8380652485712
+      }
+    }
+  },
+  "version": 0.4
+}