增加ComparingTwoFrames、右键image-to-text

Update README.md
2024-05-14 09:59:39 +08:00 · 2024-05-14 09:57:51 +08:00 · 2024-05-14 09:55:34 +08:00 · 2024-05-14 09:54:27 +08:00 · 2024-05-14 09:54:24 +08:00 · 2024-05-14 09:39:57 +08:00
19 changed files with 1865 additions and 224 deletions
@@ -1,9 +1,26 @@
-> 适配了最新版comfyui的py3.11 ，torch 2.1.2+cu121
+![](https://img.shields.io/github/release/shadowcz007/comfyui-mixlab-nodes)

+> 适配了最新版 comfyui 的 py3.11 ，torch 2.1.2+cu121
 > [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)

-#### 
-[comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab)
+
+##### `最新`：
+
+ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。模型下载后，放置到 `models/llamafile/`
+
+- 右键菜单支持 text-to-text，方便对 prompt 词补全
+
+强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)，备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
+
+- 右键菜单支持 image-to-text，使用多模态模型，多模态使用 [llava-phi-3-mini-gguf](https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main)，注意需要把llava-phi-3-mini-mmproj-f16.gguf也下载
+
+![](./assets/prompt_ai_setup.png)
+![](./assets/prompt-ai.png)
+
+
+#### `相关插件推荐`
+
+<!-- [comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab) -->

 [comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)

@@ -13,24 +30,12 @@

 <!-- [comfyui-CLIPSeg](https://github.com/shadowcz007/comfyui-CLIPSeg) -->

-#### 
-最新：ChatGPT节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
+## 🚀🚗🚚🏃 Workflow-to-APP

-Model download,move to :```models/llamafile/```
-
-强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)
-
-备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
-
-
-
-
-
-## 🚀🚗🚚🏃 Workflow-to-APP 
- 新增AppInfo节点，可以通过简单的配置，把workflow转变为一个Web APP。
- 支持多个web app 切换
- 发布为app的workflow，可以在右键里再次编辑了
- web app可以设置分类，在comfyui右键菜单可以编辑更新web app
+- 新增 AppInfo 节点，可以通过简单的配置，把 workflow 转变为一个 Web APP。
+- 支持多个 web app 切换
+- 发布为 app 的 workflow，可以在右键里再次编辑了
+- web app 可以设置分类，在 comfyui 右键菜单可以编辑更新 web app
 - 支持动态提示

 ![](./assets/微信图片_20240421205440.png)
@@ -40,7 +45,6 @@ Model download,move to :```models/llamafile/```
 - The workflow, which is now released as an app, can also be edited again by right-clicking.
 - The web app can be configured with categories, and the web app can be edited and updated in the right-click menu of ComfyUI.

-
 ![](./assets/0-m-app.png)

 ![](./assets/appinfo-readme.png)
@@ -48,69 +52,91 @@ Model download,move to :```models/llamafile/```
 ![](./assets/appinfo-2.png)

 Example:
+
 - workflow
-![APP info](./workflow/appinfo-workflow.svg)
-[text-to-image](./workflow/Text-to-Image-app.json)
+  ![APP info](./workflow/appinfo-workflow.svg)
+  [text-to-image](./workflow/Text-to-Image-app.json)

 APP-JSON:
+
 - [text-to-image](./example/Text-to-Image_3.json)
 - [image-to-image](./example/Image-to-Image_2.json)
 - text-to-text

-> 暂时支持 9 种节点作为界面上的输入节点：Load Image、VHS_LoadVideo、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
+> 暂时支持 9 种节点作为界面上的输入节点：Load Image、VHS*LoadVideo、CLIPTextEncode、PromptSlide、TextInput*、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader

 > 输出节点：PreviewImage 、SaveImage、ShowTextForGPT、VHS_VideoCombine、PromptImage

-> seed统一输入控件，支持：SamplerCustom、KSampler
+> seed 统一输入控件，支持：SamplerCustom、KSampler

-> 配套[ps插件](https://github.com/shadowcz007/comfyui-ps-plugin)
+> 配套[ps 插件](https://github.com/shadowcz007/comfyui-ps-plugin)

-> 如果遇到上传图片不成功，请检查下：局域网或者是云服务，请使用https，端口8189这个服务（ 感谢 @Damien 反馈问题）
+> 如果遇到上传图片不成功，请检查下：局域网或者是云服务，请使用 https，端口 8189 这个服务（ 感谢 @Damien 反馈问题）

 > If you encounter difficulties in uploading images, please check the following: for local network or cloud services, please use HTTPS and the service on port 8189. (Thanks to @Damien for reporting the issue.)

+## 🏃🚗🚚🚀 Real-time Design

-
-## 🏃🚗🚚🚀  Real-time Design
 > ScreenShareNode & FloatingVideoNode. Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐

 ![screenshare](./assets/screenshare.png)

 https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43e-410a-ab3a-1952b7b4e7da

-
 <!-- [ScreenShareNode](./workflow/2-screeshare.json) -->
+
 [ScreenShareNode & FloatingVideoNode](./workflow/3-FloatVideo-workflow.json)

 !! Please use the address with HTTPS (https://127.0.0.1).

-
 ### SpeechRecognition & SpeechSynthesis
+
 ![f](./assets/audio-workflow.svg)

 [Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)

 ### GPT
-> Support for calling multiple GPTs.Local LLM（llama.cpp）、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
+
+> Support for calling multiple GPTs.Local LLM（llama.cpp）、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com

 ![gpt-workflow.svg](./assets/gpt-workflow.svg)

 [workflow-5](./workflow/5-gpt-workflow.json)

+最新：ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。

-最新：ChatGPT节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
-
-Model download,move to :```models/llamafile/```
+Model download,move to :`models/llamafile/`

 强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)

 备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)

+> 如果碰到安装失败，可以尝试手动安装

+```
+../../../python_embeded/python.exe -s -m pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
+
+../../../python_embeded/python.exe -s -m pip install llama-cpp-python[server]
+
+```
+
+> [Mac](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)
+
+```
+pip uninstall llama-cpp-python -y
+CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --no-cache-dir
+pip install 'llama-cpp-python[server]'
+```
+
+```
+pip install llama-cpp-python \
+  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
+```

 ## Prompt
+
 > PromptSlide
-![](./assets/prompt_weight.png)
+> ![](./assets/prompt_weight.png)

 <!-- ![](./workflow/promptslide-appinfo-workflow.svg) -->

@@ -124,20 +150,20 @@ Model download,move to :```models/llamafile/```

 > PromptImage & PromptSimplification,Assist in simplifying prompt words, comparing images and prompt word nodes.

-> ChinesePrompt && PromptGenerate，中文prompt节点，直接用中文书写你的prompt
+> ChinesePrompt && PromptGenerate，中文 prompt 节点，直接用中文书写你的 prompt

 ![](./assets/ChinesePrompt_workflow.svg)

-
 ### Layers
+
 > A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.

 ![layers](./assets/layers-workflow.svg)

 ![poster](./assets/poster-workflow.svg)

-
 ### 3D
+
 ![](./assets/3d-workflow.png)
 ![](./assets/3d_app.png)
 [workflow](./assets/Image-to-3D_1.json)
@@ -145,44 +171,44 @@ Model download,move to :```models/llamafile/```
 ![](./assets/3dimage.png)
 [workflow](./workflow/3D-workflow.json)

-
 ### Image

 #### LoadImagesToBatch
-> Upload multiple images for batch input into the IP adapter. 
+
+> Upload multiple images for batch input into the IP adapter.

 #### LoadImagesFromLocal
-> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop. 
+
+> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.

 ![watch](./assets/4-loadfromlocal-watcher-workflow.svg)

 [workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)

 #### LoadImagesFromURL
+
 > Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.

-
 ### Style
-> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting) 
+
+> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)

 ![](./assets/VisualStylePrompting.png)

-> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy) 
-
+> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)

 ### Utils
+
 > The Color node provides a color picker for easy color selection, the Font node offers built-in font selection for use with TextImage to generate text images, and the DynamicDelayByText node allows delayed execution based on the length of the input text.

- [添加了DynamicDelayByText功能，可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
+- [添加了 DynamicDelayByText 功能，可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)

 - [Added DynamicDelayByText, enabling delayed execution based on input text length.](./workflow/audio-chatgpt-workflow.json)

- [使用CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)
+- [使用 CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)

 - [CkptNames compare the effects of different models.](./workflow/ckpts-image-workflow.json)

-
-
 ### Other Nodes

 ![main](./assets/all-workflow.svg)
@@ -190,33 +216,27 @@ Model download,move to :```models/llamafile/```

 [workflow-1](./workflow/1-workflow.json)

-
-
 > TransparentImage

 ![TransparentImage](./assets/TransparentImage.png)

-
 > FeatheredMask、SmoothMask

 Add edges to an image.

 ![FeatheredMask](./assets/FlVou_Y6kaGWYoEj1Tn0aTd4AjMI.jpg)

-
 > LaMaInpainting

 from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainting)

-
 > rembgNode

 "briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"

-*** briarmbg ***  model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
+**_ briarmbg _** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes

-
-### Improvement 
+### Improvement

 - Add "help" option to the context menu for each node.
 - Add "Nodes Map" option to the global context menu.
@@ -227,23 +247,21 @@ An improvement has been made to directly redirect to GitHub to search for missin

 ![node-not-found](./assets/node-not-found.png)

-
 ### Models

-* [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in ```models/triposr```
+- [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in `models/triposr`

-* [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main)  and place it in ```models/triposr/facebook/dino-vitb16```
+- [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main) and place it in `models/triposr/facebook/dino-vitb16`

+[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:`models/rembg`

-[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:```models/rembg```
+[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : `models/lama`

-[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : ```models/lama```
+[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :`models/clip_interrogator/Salesforce/blip-image-captioning-base`

-[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :```models/clip_interrogator/Salesforce/blip-image-captioning-base```
+[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:`models/prompt_generator/text2image-prompt-generator`

-[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:```models/prompt_generator/text2image-prompt-generator```
-
-[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:```models/prompt_generator/opus-mt-zh-en```
+[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:`models/prompt_generator/opus-mt-zh-en`

 ## Installation

@@ -259,40 +277,35 @@ git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
 Install the requirements:

 run directly:
+
 ```
 cd ComfyUI/custom_nodes/comfyui-mixlab-nodes
 install.bat
 ```

 or install the requirements using:
+
 ```
 ../../../python_embeded/python.exe -s -m pip install -r requirements.txt
 ```

 If you are using a venv, make sure you have it activated before installation and use:
+
 ```
 pip3 install -r requirements.txt
 ```

-
-
-
-
 #### Chinese community
-访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab无界社区

+访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab 无界社区

+####

-
-#### 
 File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL

-
-
-
 #### discussions:
-[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)

+[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)

 <picture>
  <source
@@ -312,4 +325,3 @@ File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
    src="https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date"
  />
 </picture>
-
@@ -11,6 +11,24 @@ import logging
 from comfy.cli_args import args
 python = sys.executable

+#修复 sys.stdout.isatty()  object has no attribute 'isatty'
+try:
+    sys.stdout.isatty()
+except:
+    print('#fix sys.stdout.isatty')
+    sys.stdout.isatty = lambda: False
+
+llama_port=None
+llama_model=""
+llama_chat_format=""
+
+try:
+    from .nodes.ChatGPT import get_llama_models,get_llama_model_path,llama_cpp_client
+    llama_cpp_client("")
+
+except:
+    print("##nodes.ChatGPT ImportError")
+

 from server import PromptServer

@@ -43,7 +61,7 @@ def is_installed(package, package_overwrite=None):
            print(f"Couldn't install\nCommand: {command}\nError code: {result.returncode}")
    else:
        print(package+'## OK')
-        
+     
 try:
    import OpenSSL
 except ImportError:
@@ -437,19 +455,26 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):

        ssl_context = None
        scheme = "http"
-        # 跟着本体修改
-        if args.tls_keyfile and args.tls_certfile:
+        try:
+            # 跟着本体修改
+            if args.tls_keyfile and args.tls_certfile:
                scheme = "https"
                ssl_context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_SERVER, verify_mode=ssl.CERT_NONE)
                ssl_context.load_cert_chain(certfile=args.tls_certfile,
-                                keyfile=args.tls_keyfile)
-        else:
-            # 如果没传，则自动创建
+                                    keyfile=args.tls_keyfile)
+            else:
+                # 如果没传，则自动创建
+                import ssl
+                crt, key = create_for_https()
+                ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
+                ssl_context.load_cert_chain(crt, key)
+        except:
            import ssl
            crt, key = create_for_https()
            ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
            ssl_context.load_cert_chain(crt, key)

+
        success = False
        for i in range(11):  # 尝试最多11次
            if await check_port_available(address, http_port + 1 + i):
@@ -480,10 +505,14 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
            # print("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))

        if call_on_start is not None:
-            if scheme=='https':
-                call_on_start(scheme,address, https_port)
-            else:
-                call_on_start(scheme,address, http_port)
+            try:
+                if scheme=='https':
+                    call_on_start(scheme,address, https_port)
+                else:
+                    call_on_start(scheme,address, http_port)
+            except:
+                call_on_start(address,http_port)
+            

    except Exception as e:
        print(f"Error starting the server: {e}")
@@ -494,8 +523,6 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
        # webbrowser.open(f"https://{address}")
        # webbrowser.open(f"http://{address}:{port}")

-
-
 PromptServer.start=new_start

 # 创建路由表
@@ -590,13 +617,19 @@ async def nodes_map_hander(request):
 async def get_checkpoints(request):
    data = await request.json()
    t="checkpoints"
+    names=[]
    try:
        t=data['type']
+        names = folder_paths.get_filename_list(t)
    except Exception as e:
        print('/mixlab/folder_paths',False,e)
-    
-    names = folder_paths.get_filename_list(t)

+    try:
+        if data['type']=='llamafile':
+            names=get_llama_models()
+    except:
+        print("llamafile none")
+    
    return web.json_response({"names":names,"types":list(folder_paths.folder_names_and_paths.keys())})


@@ -617,20 +650,130 @@ async def post_prompt_result(request):
    
    return web.json_response({"result":res})

-# 扩展api接口
-# from server import PromptServer
-# from aiohttp import web

-# @routes.post('/ws_image')
-# async def my_hander_method(request):
-#     post = await request.post()
-#     x = post.get("something")
-#     return web.json_response({})
+async def start_local_llm(data):
+    global llama_port,llama_model,llama_chat_format
+    if llama_port and llama_model and llama_chat_format:
+        return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
+
+    import threading
+    import uvicorn
+    from llama_cpp.server.app import create_app
+    from llama_cpp.server.settings import (
+                Settings,
+                ServerSettings,
+                ModelSettings,
+                ConfigFileSettings,
+            )
+    
+    if not "model" in data and "model_path" in data:
+        data['model']= os.path.basename(data["model_path"])
+        model=data["model_path"]
+
+    elif "model" in data:
+        model=get_llama_model_path(data['model'])
+
+    n_gpu_layers=-1
+
+    if "n_gpu_layers" in data:
+        n_gpu_layers=data['n_gpu_layers']
+
+
+    chat_format="chatml"
+
+    model_alias=os.path.basename(model)
+
+    # 多模态
+    clip_model_path=None
+    
+    prefix = "llava-phi-3-mini"
+    file_name = prefix+"-mmproj-"
+    if model_alias.startswith(prefix):
+        for file in os.listdir(os.path.dirname(model)):
+            if file.startswith(file_name):
+                clip_model_path=os.path.join(os.path.dirname(model),file)
+                chat_format='llava-1-5'
+    print('#clip_model_path',chat_format,clip_model_path)
+
+
+    address="127.0.0.1"
+    port=9090
+    success = False
+    for i in range(11):  # 尝试最多11次
+        if await check_port_available(address, port + i):
+            port = port + i
+            success = True
+            break
+
+    if success == False:
+        return {"port":None,"model":""}
+    
+    
+    server_settings=ServerSettings(host=address,port=port)
+
+    app = create_app(
+                server_settings=server_settings,
+                model_settings=[
+                    ModelSettings(
+                    model=model,
+                    model_alias=os.path.basename(model),
+                    n_gpu_layers=n_gpu_layers,
+                    n_ctx=4098,
+                    chat_format=chat_format,
+                    embedding=False,
+                    clip_model_path=clip_model_path
+                    )])
+
+    def run_uvicorn():
+        uvicorn.run(
+                app,
+                host=os.getenv("HOST", server_settings.host),
+                port=int(os.getenv("PORT", server_settings.port)),
+                ssl_keyfile=server_settings.ssl_keyfile,
+                ssl_certfile=server_settings.ssl_certfile,
+            )
+
+    # 创建一个子线程
+    thread = threading.Thread(target=run_uvicorn)
+
+    # 启动子线程
+    thread.start()
+
+    llama_port=port
+    llama_model=data['model']
+    llama_chat_format=chat_format
+
+    return  {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
+
+# llam服务的开启
+@routes.post('/mixlab/start_llama')
+async def my_hander_method(request):
+    data =await request.json()
+    # print(data)
+    try:
+        result=await start_local_llm(data)
+    except:
+        result={
+            {"port":None,"model":"","llama_cpp_error":True}
+        }
+        print('start_local_llm error')
+
+    return web.json_response(result)
+
+# 重启服务
+@routes.post('/mixlab/re_start')
+def re_start(request):
+    try:
+        sys.stdout.close_log()
+    except Exception as e:
+        pass
+    return os.execv(sys.executable, [sys.executable] + sys.argv)
+


 # 导入节点
 from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
-from .nodes.ImageNode import LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
+from .nodes.ImageNode import ComparingTwoFrames,LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
 # from .nodes.Vae import VAELoader,VAEDecode
 from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo

@@ -691,6 +834,7 @@ NODE_CLASS_MAPPINGS = {
    # "VAELoaderConsistencyDecoder":VAELoader,
    "SaveImageToLocal":SaveImageToLocal,
    "SaveImageAndMetadata_":SaveImageAndMetadata,
+    "ComparingTwoFrames_":ComparingTwoFrames,
    # "VAEDecodeConsistencyDecoder":VAEDecode,
    "ScreenShare":ScreenShareNode,
    "FloatingVideo":FloatingVideo,
@@ -745,6 +889,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
    "IntNumber":"Int Input ♾️MixlabApp",
    "ImagesPrompt_":"Images Input ♾️MixlabApp",
    "SaveImageAndMetadata_":"Save Image Output ♾️MixlabApp",
+    "ComparingTwoFrames_":"Comparing Two Frames ♾️MixlabApp",
    "ResizeImageMixlab":"Resize Image ♾️Mixlab",
    "RandomPrompt": "Random Prompt ♾️Mixlab",
    "PromptImage":"Output Prompt and Image ♾️Mixlab",
@@ -836,4 +981,4 @@ try:
 except Exception as e:
    logging.info('RembgNode_.available False' )

-logging.info('\033[93m -------------- \033[0m')
+logging.info('\033[93m -------------- \033[0m')
@@ -10,6 +10,12 @@ if exist "%python_exec%" (
    for /f "delims=" %%i in (%requirements_txt%) do (
        %python_exec% -s -m pip install "%%i" -i https://pypi.tuna.tsinghua.edu.cn/simple
    )
+
+    %python_exec% -s -m pip install --upgrade --force llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
+
+    %python_exec% -s -m pip install --upgrade --force llama-cpp-python[server]
+    
+
 ) else (
    echo Installing with system Python
    for /f "delims=" %%i in (%requirements_txt%) do (
@@ -91,10 +91,16 @@ def ZhipuAI_client(key):
 def phi_sort(lst):
    return sorted(lst, key=lambda x: x.lower().count('phi'), reverse=True)

+def get_llama_path():
+    try:
+        return folder_paths.get_folder_paths('llamafile')[0]
+    except:
+        return os.path.join(folder_paths.models_dir, "llamafile")
+
 def get_llama_models():
    res=[]

-    model_path=os.path.join(folder_paths.models_dir, "llamafile")
+    model_path=get_llama_path()
    if os.path.exists(model_path):
        files = os.listdir(model_path)
        for file in files:
@@ -106,7 +112,7 @@ def get_llama_models():
 llama_modes_list=get_llama_models()

 def get_llama_model_path(file_name):
-    model_path=os.path.join(folder_paths.models_dir, "llamafile")
+    model_path=get_llama_path()
    mp=os.path.join(model_path,file_name)
    return mp

@@ -129,6 +135,12 @@ def llama_cpp_client(file_name):
            if result.returncode == 0:
                print("#install success")
                from llama_cpp import Llama
+
+                subprocess.run([sys.executable, '-s', '-m', 'pip', 
+                                     'install', 
+                                     'llama-cpp-python[server]'
+                                     ], capture_output=True, text=True)
+
            else:
                print("#install error")
            
@@ -137,14 +149,15 @@ def llama_cpp_client(file_name):
    except:
        print("#install llama-cpp-python error")

-    mp=get_llama_model_path(file_name)
-    # file_name=get_llama_models()[0]
-    # model_path=os.path.join(folder_paths.models_dir, "llamafile")
-    # mp=os.path.join(model_path,file_name)
+    if file_name:
+        mp=get_llama_model_path(file_name)
+        # file_name=get_llama_models()[0]
+        # model_path=os.path.join(folder_paths.models_dir, "llamafile")
+        # mp=os.path.join(model_path,file_name)

-    llm = Llama(model_path=mp, chat_format="chatml")
+        llm = Llama(model_path=mp, chat_format="chatml",n_gpu_layers=-1,n_ctx=512)

-    return llm
+        return llm

    

@@ -70,14 +70,20 @@ def load_caption_model(model_path,config,t='blip-base'):
    return (caption_model,caption_processor)
    

+def get_clip_interrogator_path():
+    try:
+        return folder_paths.get_folder_paths('clip_interrogator')[0]
+    except:
+        return os.path.join(folder_paths.models_dir, "clip_interrogator")

-caption_model_path=os.path.join(folder_paths.models_dir, "clip_interrogator/Salesforce/blip-image-captioning-base")
+
+cache_path=get_clip_interrogator_path()
+
+caption_model_path=os.path.join(cache_path, "Salesforce/blip-image-captioning-base")
 if not os.path.exists(caption_model_path):
    print(f"## clip_interrogator_model not found: {caption_model_path}, pls download from https://huggingface.co/Salesforce/blip-image-captioning-base")
    caption_model_path='Salesforce/blip-image-captioning-base'

-cache_path=os.path.join(folder_paths.models_dir, "clip_interrogator")
-

 # Tensor to PIL
 def tensor2pil(image):
@@ -29,23 +29,122 @@ def opencv_to_pil(image):
    return pil_image


-def composite_images(foreground, background, mask,is_multiply_blend=False):
+def composite_images(foreground, background, mask,is_multiply_blend=False,position="overall"):
    width,height=foreground.size
 
    bg_image=background
+
+    bwidth,bheight=bg_image.size
+
    # 按z-index排序
-    layer = {
-            "x":0,
-            "y":0,
-            "width":width,
-            "height":height,
-            "z_index":88,
-            "scale_option":'overall',
-            "image":foreground,
-            "mask":mask
-    }
+    if position=="overall":
+        layer = {
+                "x":0,
+                "y":0,
+                "width":bwidth,
+                "height":bheight,
+                "z_index":88,
+                "scale_option":'overall',
+                "image":foreground,
+                "mask":mask
+        }
+
+    elif position=='center_bottom':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":int(bwidth*0.75*0.5),
+                "y":bheight-new_height-24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
+
+    elif position=='right_bottom':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":bwidth-int(bwidth*0.25)-24,
+                "y":bheight-new_height-24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
+
+
+    elif position=='center_top':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":int( bwidth*0.75*0.5),
+                "y":24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
+
+    elif position=='right_top':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":bwidth-int(bwidth*0.25)-24,
+                "y":24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
+    elif position=='left_top':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":24,
+                "y":24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
+    elif position=='left_bottom':
+        
+        scale = int(bwidth*0.25) / width
+        new_height = int(height * scale)
+
+        layer = {
+                "x":24,
+                "y":bheight-new_height-24,
+                "width":int(bwidth*0.25),
+                "height":int(bheight*0.25),
+                "z_index":88,
+                "scale_option":'width',
+                "image":foreground,
+                "mask":mask
+        }
                
-    width, height = bg_image.size
+    # width, height = bg_image.size

    layer_image=layer['image']
    layer_mask=layer['mask']
@@ -729,6 +828,27 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,
        # 整体缩放
        layer_image = layer_image.resize((width, height))

+    elif scale_option == "longest":
+        original_width, original_height = layer_image.size
+        if original_width > original_height:
+            new_width=width
+            scale = width / original_width
+            new_height = int(original_height * scale)
+            x=0
+            y=int((height-new_height)*0.5)
+        else:
+            new_height=height
+            scale = height / original_height
+            new_width = int(original_height * scale)
+            x=int((width-new_width)*0.5)
+            y=0
+    # elif side == "shortest":
+    #         if width < height:
+    #              
+    #         else:
+    #              
+    
+
    # 调整mask的大小
    nw, nh = layer_image.size
    mask = mask.resize((nw, nh))
@@ -752,8 +872,9 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,
        bg_image=multiply_blend(bg_image_white,bg_image)
        bg_image=bg_image.convert("RGBA")
    else:
-        transparent_img = Image.new("RGBA",layer_image.size, (0, 0, 0, 0))
+        transparent_img = Image.new("RGBA",layer_image.size, (255, 255, 255, 0))
        transparent_img.paste(layer_image,(0, 0), mask)
+        # transparent_img.save('test.png')
        bg_image.paste(transparent_img, (x, y), transparent_img)


@@ -803,6 +924,31 @@ def resize_image(layer_image, scale_option, width, height,color="white"):
        resized_image = resized_image.convert("RGB")
        resized_image=resize_2(resized_image)
        return resized_image
+    elif scale_option == "longest":
+    #暂时不用，  
+        if original_width > original_height:
+            new_width=width
+            scale = width / original_width
+            new_height = int(original_height * scale)
+            x=0
+            y=int((new_height-height)*0.5)
+            resized_image = Image.new("RGB", (new_width, new_height), color=color)
+            resized_image.paste(layer_image.resize((new_width, new_height)), (x,y))
+            resized_image = resized_image.convert("RGB")
+            resized_image=resize_2(resized_image)
+            return resized_image
+        else:
+            new_height=height
+            scale = height / original_height
+            new_width = int(original_height * scale)
+            x=int((new_width-width)*0.5)
+            y=0
+            resized_image = Image.new("RGB", (new_width, new_height), color=color)
+            resized_image.paste(layer_image.resize((new_width, new_height)), (x,y))
+            resized_image = resized_image.convert("RGB")
+            resized_image=resize_2(resized_image)
+            return resized_image
+
    
    layer_image=resize_2(layer_image)
    return layer_image
@@ -1704,14 +1850,14 @@ class CompositeImages:
    def INPUT_TYPES(s):
        return {
            "required": { 
-                    "foreground": ("IMAGE",),
+                    "foreground": (any_type,),
                    "mask":("MASK",),
                    "background": ("IMAGE",),
                    },
             "optional":{
                           
                  "is_multiply_blend":  ("BOOLEAN", {"default": False}),
-                            
+                  "position":  (['overall',"center_bottom","center_top","right_bottom","left_bottom","right_top","left_top"],),    
                    }
                }
    
@@ -1724,11 +1870,11 @@ class CompositeImages:

    # OUTPUT_IS_LIST = (True,)

-    def run(self, foreground,mask,background,is_multiply_blend):
+    def run(self, foreground,mask,background,is_multiply_blend,position):
        foreground= tensor2pil(foreground)
        mask= tensor2pil(mask)
        background= tensor2pil(background)
-        res=composite_images(foreground,background,mask,is_multiply_blend)
+        res=composite_images(foreground,background,mask,is_multiply_blend,position)
        
        return (pil2tensor(res),)

@@ -2837,6 +2983,65 @@ class SaveImageAndMetadata:

        return { "ui": { "images": results } }

+class ComparingTwoFrames:
+    def __init__(self):
+        self.output_dir = folder_paths.get_output_directory()
+        self.type = "output"
+        self.prefix_append = "ComparingTwoFrames"
+        self.compress_level = 4
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": 
+                    {"before_image": ("IMAGE", ),
+                    "after_image": ("IMAGE", )
+                     }, 
+                }
+
+    RETURN_TYPES = ()
+    FUNCTION = "comparingImages"
+
+    OUTPUT_NODE = True
+
+    CATEGORY = "♾️Mixlab/Output"
+
+    def comparingImages(self, before_image,after_image):
+        filename_prefix = self.prefix_append
+        full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
+            filename_prefix, self.output_dir, after_image[0].shape[1], after_image[0].shape[0])
+        
+        bresults = list()
+        
+        for bimage in before_image:
+            i = 255. * bimage.cpu().numpy()
+            img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
+         
+            file = f"{filename}_{counter:05}_.png"
+            img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
+            bresults.append({
+                "filename": file,
+                "subfolder": subfolder,
+                "type": self.type
+            })
+            counter += 1
+
+
+        results = list()
+        for aimage in after_image:
+            i = 255. * aimage.cpu().numpy()
+            img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
+         
+            file = f"{filename}_{counter:05}_.png"
+            img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
+            results.append({
+                "filename": file,
+                "subfolder": subfolder,
+                "type": self.type
+            })
+            counter += 1
+
+        return { "ui": { "after_images": results,"before_images":bresults } }
+
 class ImageColorTransfer:
    @classmethod
    def INPUT_TYPES(s):
@@ -42,8 +42,13 @@ else:
    _available=True


- 
-llma_model_path=os.path.join(folder_paths.models_dir, "lama/big-lama.pt")
+def get_lama_path():
+    try:
+        return folder_paths.get_folder_paths('lama')[0]
+    except:
+        return os.path.join(folder_paths.models_dir, "lama")
+
+llma_model_path=os.path.join(get_lama_path(), "big-lama.pt")
 if not os.path.exists(llma_model_path):
    os.environ['LAMA_MODEL']=''
    print(f"## lama torchscript model not found: {llma_model_path},pls download from https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt")
@@ -18,19 +18,25 @@ from lark import Lark, Transformer, v_args
 global _available
 _available=True

+def get_text_generator_path():
+    try:
+        return folder_paths.get_folder_paths('prompt_generator')[0]
+    except:
+        return os.path.join(folder_paths.models_dir, "prompt_generator")

-text_generator_model_path=os.path.join(folder_paths.models_dir, "prompt_generator/text2image-prompt-generator")
+prompt_generator=get_text_generator_path()
+
+text_generator_model_path=os.path.join(prompt_generator, "text2image-prompt-generator")
 if not os.path.exists(text_generator_model_path):
    print(f"## text_generator_model not found: {text_generator_model_path}, pls download from https://huggingface.co/succinctly/text2image-prompt-generator/tree/main")
    text_generator_model_path='succinctly/text2image-prompt-generator'

-zh_en_model_path=os.path.join(folder_paths.models_dir, "prompt_generator/opus-mt-zh-en")
+zh_en_model_path=os.path.join(prompt_generator, "opus-mt-zh-en")
 if not os.path.exists(zh_en_model_path):
    print(f"## zh_en_model not found: {zh_en_model_path}, pls download from https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main")
    zh_en_model_path='Helsinki-NLP/opus-mt-zh-en'


-
 def is_installed(package):
    try:
        spec = importlib.util.find_spec(package)
@@ -247,10 +247,10 @@

        .card textarea {
            width: 100%;
-            /* height: 200px; */
+            height:'fit-content';
            /* min-width: 300px; */
            margin-top: 12px;
-            resize: vertical;
+            resize: none;
            overflow: hidden;
        }

@@ -434,7 +434,7 @@
            /* 角落颜色 */
        }

-
+/* 
        .dynamic_prompt::after {
            content: attr(title);
            position: absolute;
@@ -442,12 +442,11 @@
            padding: 4px;
            padding-left: 25px;
            border-radius: 4px;
-        }
+        } */

-        summary{
+        summary {
            user-select: none;
        }
-        
    </style>
    <!-- <script src="../../../scripts/api.js" type="module"></script> -->
    <link href="/extensions/comfyui-mixlab-nodes/lib/photoswipe.min.css" rel="stylesheet">
@@ -953,6 +952,10 @@


        function createOutputs(outputData, link) {
+            const url = new URL(window.location.href);
+            const params = new URLSearchParams(url.search);
+            const innerApp = params.get("innerApp");
+
            const container = document.createElement('div');
            container.className = "output";

@@ -961,11 +964,11 @@

            const copyHTML = document.createElement('button');
            copyHTML.innerText = 'copy as html'
-            action.appendChild(copyHTML)
+            if (!innerApp) action.appendChild(copyHTML)

            const copyImage = document.createElement('button');
            copyImage.innerText = 'copy image'
-            action.appendChild(copyImage)
+            if (!innerApp) action.appendChild(copyImage)
            copyImage.style.marginLeft = '18px';

            let isURL = false;
@@ -1067,18 +1070,19 @@
                    "Image Save",
                    "SaveImageAndMetadata_",
                    "TransparentImage"].includes(node.class_type)) {
-                    console.log('output#image', node)
+
+                    const url = node.options?.defaultImage || window._appData?.icon || base64Df;
                    let a = document.createElement('a');
                    a.id = `output_${node.id}`
                    a.setAttribute('data-pswp-width', "200");
                    a.setAttribute('data-pswp-height', "200");
                    a.setAttribute('target', "_blank");
-                    a.setAttribute('href', base64Df);
+                    a.setAttribute('href', url);
                    a.setAttribute('title', node.title);

                    let img = new Image();
                    // img;
-                    img.src = window._appData?.icon || base64Df;
+                    img.src = url;
                    a.appendChild(img)
                    output_card.appendChild(a);
                    isShowImageFn = true;
@@ -2272,7 +2276,7 @@
            // leftDiv.appendChild(des);
            leftDiv.appendChild(statusDiv);
            leftDiv.appendChild(input1);
-            
+
            if (typeof (data.data) == 'object') mainDiv.appendChild(submitButton);

            rightDiv.appendChild(output);
@@ -2630,7 +2634,7 @@
            ui.title.update(appData.name || 'Mixlab APP');

            // 更新应用图标
-            ui.icon.update(appData.icon || base64Df);
+            ui.icon.update(appData.icon || appData.output[0]?.options?.defaultImage || base64Df);

            ui.des.update(appData.description || '-');

@@ -207,9 +207,27 @@ async function extractInputAndOutputData (
        // input.push()
      }
      if (outputIds.includes(id)) {
+        let options = {}
+        //输出的默认图
+        if (
+          node.type === 'SaveImageAndMetadata_' &&
+          app.graph.getNodeById(id).imgs
+        ) {
+          // SaveImageAndMetadata_的默认图，转为base64
+          let imgurl = app.graph.getNodeById(id).imgs[0].src
+
+          options.defaultImage = await drawImageToCanvas(imgurl, 512)
+          console.log('#SaveImageAndMetadata_的默认图', options)
+        }
+
        // let node = app.graph.getNodeById(id)
        // output.push()
-        output[outputIds.indexOf(id)] = { ...data[id], title: node.title, id }
+        output[outputIds.indexOf(id)] = {
+          ...data[id],
+          title: node.title,
+          id,
+          options
+        }
      }

      if (
@@ -0,0 +1,106 @@
+async function* completion (url, messages, controller) {
+  let data = {
+    model: 'gpt-3.5-turbo-16k',
+    messages,
+    temperature: 0.6,
+    stream: true
+  }
+  // if (imageNode) {
+  //   data = { ...data, image_data: [imageNode] }
+  // }
+
+  // let controller = new AbortController()
+
+  let response = await fetch(url, {
+    method: 'POST',
+    body: JSON.stringify(data),
+    headers: {
+      Connection: 'keep-alive',
+      'Content-Type': 'application/json',
+      Accept: 'text/event-stream'
+    },
+    signal: controller.signal
+  })
+
+  const reader = response.body.getReader()
+  const decoder = new TextDecoder()
+
+  let content = ''
+  let leftover = '' // Buffer for partially read lines
+
+  try {
+    let cont = true
+    while (cont) {
+      let result = await reader.read()
+      if (result.done) {
+        break
+      }
+
+      // Add any leftover data to the current chunk of data
+      const text = leftover + decoder.decode(result.value)
+
+      // Check if the last character is a line break
+      const endsWithLineBreak = text.endsWith('\n')
+
+      // Split the text into lines
+      let lines = text.split('\n')
+
+      // If the text doesn't end with a line break, then the last line is incomplete
+      // Store it in leftover to be added to the next chunk of data
+      if (!endsWithLineBreak) {
+        leftover = lines.pop()
+      } else {
+        leftover = '' // Reset leftover if we have a line break at the end
+      }
+
+      // Parse all sse events and add them to result
+      const regex = /^(\S+):\s(.*)$/gm
+      for (const line of lines) {
+        const match = regex.exec(line)
+        if (match) {
+          result[match[1]] = match[2]
+          // since we know this is llama.cpp, let's just decode the json in data
+          if (result.data) {
+            result.data = JSON.parse(result.data)
+            // console.log('#result.data',result.data)
+
+            content += result.data.choices[0].delta?.content || ''
+
+            // yield
+            yield result
+
+            // if we got a stop token from server, we will break here
+            if (result.data.choices[0].finish_reason == 'stop') {
+              if (result.data.generation_settings) {
+                // generation_settings = result.data.generation_settings;
+              }
+              cont = false
+              break
+            }
+          }
+        }
+      }
+    }
+  } catch (e) {
+    console.error('llama error: ', e)
+    throw e
+  } finally {
+    controller.abort()
+  }
+
+  return content
+  // return (await response.json()).content
+}
+
+export async function completion_ (url, messages, controller, callback) {
+  let request = await completion(url, messages, controller)
+  for await (const chunk of request) {
+    let content = chunk.data.choices[0].delta.content || ''
+    if (chunk.data.choices[0].role == 'assistant') {
+      //开始
+      content = ''
+    }
+
+    if (callback) callback(content)
+  }
+}
@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
 const repoOwner = 'shadowcz007' // 替换为仓库的所有者
 const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称

-const version = 'v0.24.0'
+const version = 'v0.26.0'

 fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
  .then(response => response.json())
@@ -705,7 +705,9 @@ app.registerExtension({

  async beforeRegisterNodeDef (nodeType, nodeData, app) {
    if (nodeType.comfyClass == 'LoadImagesToBatch') {
+
      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+
      nodeType.prototype.onNodeCreated = function () {
        orig_nodeCreated?.apply(this, arguments)

@@ -841,3 +843,150 @@ app.registerExtension({
    }
  }
 })
+
+// 如何引入css
+app.registerExtension({
+  name: 'Mixlab.output.ComparingTwoFrames_',
+  init () {
+    $el('link', {
+      rel: 'stylesheet',
+      href: '/extensions/comfyui-mixlab-nodes/lib/juxtapose.css',
+      parent: document.head
+    })
+
+    $el('style', {
+      textContent: `
+      .juxtapose-name{
+        display: none!important; 
+      }
+			`,
+      parent: document.body
+    })
+  },
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'ComparingTwoFrames_') {
+      const onNodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        const r = onNodeCreated
+        ? onNodeCreated.apply(this, arguments)
+        : undefined
+
+        this.size = [400, this.size[1]]
+        console.log('##onNodeCreated', this)
+        const widget = {
+          type: 'div',
+          name: 'preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, 400, 44, node.size[1])
+            )
+          },
+          serialize: false
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+        this.addCustomWidget(widget)
+        this.serialize_widgets = true //需要保存参数
+
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+
+       return r
+
+      }
+
+     
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        console.log('##onExecuted', this, message)
+
+        this.widgets[0].div.id = 'mix_comparingtowframes_' + this.id
+
+        let after_image = message.after_images[0]
+        let before_image = message.before_images[0]
+
+        after_image = `${window.location.protocol}//${
+          window.location.hostname
+        }:${window.location.port}/view?filename=${encodeURIComponent(
+          after_image.filename
+        )}&type=${after_image.type}&subfolder=${encodeURIComponent(
+          after_image.subfolder
+        )}&t=${+new Date()}`
+
+        before_image = `${window.location.protocol}//${
+          window.location.hostname
+        }:${window.location.port}/view?filename=${encodeURIComponent(
+          before_image.filename
+        )}&type=${before_image.type}&subfolder=${encodeURIComponent(
+          before_image.subfolder
+        )}&t=${+new Date()}`
+
+        this.widgets[0].div.innerHTML = ''
+
+        let slider = new juxtapose.JXSlider(
+          '#mix_comparingtowframes_' + this.id,
+          [
+            {
+              src: before_image,
+              label: 'Before'
+            },
+            {
+              src: after_image,
+              label: 'After'
+            }
+          ],
+          {
+            animate: true,
+            showLabels: true,
+            showCredits: false,
+            startingPosition: '50%',
+            makeResponsive: false
+          }
+        )
+
+        this.widgets_values = [
+          {
+            src: before_image,
+            label: 'Before'
+          },
+          {
+            src: after_image,
+            label: 'After'
+          }
+        ]
+        this.size=[this.size[0],300]
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // console.log('##loadedGraphNode', node)
+    if (node.type === 'ComparingTwoFrames_') {
+      // node.widgets[0].div.id = 'mix_comparingtowframes_' + node.id
+      // if (node.widgets_values && node.widgets_values[0]) {
+      //   node.widgets[0].div.innerHTML = ''
+
+      //   let slider = new juxtapose.JXSlider(
+      //     '#mix_comparingtowframes_' + node.id,
+      //     node.widgets_values,
+      //     {
+      //       animate: true,
+      //       showLabels: true,
+      //       showCredits: false,
+      //       startingPosition: '50%',
+      //       makeResponsive: false
+      //     }
+      //   )
+      // }
+    }
+  }
+})
@@ -9,6 +9,148 @@ import {

 import { smart_init, addSmartMenu } from './smart_connect.js'

+import { completion_ } from './chat.js'
+
+function showTextByLanguage (key, json) {
+  // 获取浏览器语言
+  var language = navigator.language
+  // 判断是否为中文
+  if (
+    language.indexOf('zh') !== -1 ||
+    (language.indexOf('cn') !== -1 && json[key])
+  ) {
+    return json[key]
+  } else {
+    return key
+  }
+}
+
+//系统prompt
+const systemPrompt = `You are a prompt creator, your task is to create prompts for the user input request, the prompts are image descriptions that include keywords for (an adjective, type of image, framing/composition, subject, subject appearance/action, environment, lighting situation, details of the shoot/illustration, visuals aesthetics and artists), brake keywords by comas, provide high quality, non-verboose, coherent, brief, concise, and not superfluous prompts, the subject from the input request must be included verbatim on the prompt,the prompt is english`
+
+if (!localStorage.getItem('_mixlab_system_prompt')) {
+  localStorage.setItem('_mixlab_system_prompt', systemPrompt)
+}
+
+// 获取llama 模型
+async function get_llamafile_models () {
+  try {
+    const response = await fetch('/mixlab/folder_paths', {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        type: 'llamafile'
+      })
+    })
+
+    const data = await response.json()
+    console.log(data)
+    return data.names
+  } catch (error) {
+    console.error(error)
+  }
+}
+// 运行llama
+async function start_llama (model = 'Phi-3-mini-4k-instruct-Q5_K_S.gguf') {
+  let n_gpu_layers = -1
+  try {
+    n_gpu_layers = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
+  } catch (error) {}
+
+  try {
+    const response = await fetch('/mixlab/start_llama', {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json'
+      },
+      body: JSON.stringify({
+        model,
+        n_gpu_layers
+      })
+    })
+
+    const data = await response.json()
+    if (data.llama_cpp_error) {
+      return
+    }
+
+    return {
+      url: `http://${window.location.hostname}:${data.port}`,
+      model: data.model,
+      chat_format: data.chat_format
+    }
+  } catch (error) {
+    console.error(error)
+  }
+}
+
+function resizeImage (base64Image) {
+  var img = new Image()
+  var canvas = document.createElement('canvas')
+  var ctx = canvas.getContext('2d')
+  return new Promise((res, rej) => {
+    img.onload = function () {
+      // 等比例缩放图片
+      var width = img.width
+      var height = img.height
+      var max_width = 768
+      if (width > max_width) {
+        height *= max_width / width
+        width = max_width
+      }
+
+      // 设置canvas尺寸
+      canvas.width = width
+      canvas.height = height
+
+      // 在canvas上绘制图片
+      ctx.drawImage(img, 0, 0, width, height)
+
+      // 将canvas转换为base64图片数据
+      var canvasData = canvas.toDataURL()
+      res(canvasData) //  canvas转换后的base64图片数据
+    }
+
+    img.src = base64Image
+  })
+}
+
+// 菜单入口
+async function createMenu () {
+  const menu = document.querySelector('.comfy-menu')
+  const separator = document.createElement('div')
+  separator.style = `margin: 20px 0px;
+  width: 100%;
+  height: 1px;
+  background: var(--border-color);
+  `
+  menu.append(separator)
+
+  if (!menu.querySelector('#mixlab_chatbot_by_llamacpp')) {
+    const appsButton = document.createElement('button')
+    appsButton.id = 'mixlab_chatbot_by_llamacpp'
+    appsButton.textContent = '♾️Mixlab'
+
+    // appsButton.onclick = () =>
+    appsButton.onclick = async () => {
+      if (window._mixlab_llamacpp) {
+        //显示运行的模型
+        createModelsModal([
+          window._mixlab_llamacpp.url,
+          window._mixlab_llamacpp.model
+        ])
+      } else {
+        let ms = await get_llamafile_models()
+        ms = ms.filter(m => !m.match('-mmproj-'))
+        if (ms.length > 0) createModelsModal(ms)
+      }
+    }
+    menu.append(appsButton)
+  }
+}
+
 let isScriptLoaded = {}

 function loadExternalScript (url) {
@@ -299,8 +441,9 @@ async function get_my_app (filename = null, category = '') {
    data = []

    for (const res of result.data) {
-      let { app, workflow } = res.data;
-      if (app?.filename) data.push({
+      let { app, workflow } = res.data
+      if (app?.filename)
+        data.push({
          ...app,
          data: workflow,
          date: res.date
@@ -344,6 +487,33 @@ injectCSS(`::-webkit-scrollbar {
  width: 2px;
 }

+#mixlab_chatbot_by_llamacpp{
+  font-size:14px
+}
+
+#mixlab_chatbot_by_llamacpp::before {
+  content: attr(title);
+  position: absolute;
+  margin-top: 24px;
+  font-size: 10px;
+}
+
+.mix_tag{
+  padding:8px;cursor: pointer;font-size: 14px;
+    color: var(--input-text);
+    background-color: var(--comfy-input-bg);
+    border-radius: 8px;
+    border-color: var(--border-color);
+    border-style: solid;
+    margin-top: 2px;
+    margin-bottom: 14px;
+}
+
+.mix_tag:hover{
+  background-color: #101c19;
+  color: aquamarine;
+}
+
@keyframes loading_mixlab {
  0% {
    background-color: green;
@@ -494,13 +664,28 @@ app.showMissingNodesError = async function (
  // console.log('#nodesMap', nodesMap)
  // console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
  this.ui.dialog.show(
-    `When loading the graph, the following node types were not found: <ul>${missingNodeGithub(
-      missingNodeTypes,
-      nodesMap
-    ).join('')}</ul>${
-      hasAddedNodes
-        ? 'Nodes that have failed to load will show as red on the graph.'
-        : ''
+    `<a style="color: white;
+    font-size: 18px;
+    font-weight: 800;
+    letter-spacing: 2px;
+    font-family: sans-serif;
+  }"
+  href="https://discord.gg/cXs9vZSqeK"  target="_blank">${showTextByLanguage(
+    'Welcome to Mixlab nodes discord, seeking help.',
+    {
+      'Welcome to Mixlab nodes discord, seeking help.':
+        '寻求帮助，加入Mixlab nodes交流频道'
+    }
+  )}</a><br><br>${showTextByLanguage(
+      'When loading the graph, the following node types were not found:',
+      {
+        'When loading the graph, the following node types were not found:':
+          '缺少以下节点：'
+      }
+    )}
+  
+   <ul>${missingNodeGithub(missingNodeTypes, nodesMap).join('')}</ul>${
+      hasAddedNodes ? '' : ''
    }`
  )
  this.logging.addEntry('Comfy.App', 'warn', {
@@ -586,13 +771,245 @@ async function fetchReadmeContent (url) {
  }
 }

+function createModelsModal (models) {
+  var div =
+    document.querySelector('#model-modal') || document.createElement('div')
+  div.id = 'model-modal'
+  div.innerHTML = ''
+  div.style.cssText = `
+    width: 100%;
+    z-index: 9990;
+    height: 100vh;
+    display: flex;
+    color: var(--descrip-text);
+    position: fixed;
+    top: 0;
+    left: 0;
+    background: #000000a8;
+    `
+
+  var modal = document.createElement('div')
+
+  div.addEventListener('click', e => {
+    e.stopPropagation()
+    div.remove()
+  })
+
+  div.appendChild(modal)
+  modal.classList.add('modal-body')
+  // Set modal styles
+  modal.style.cssText = `
+  color: var(--descrip-text);
+    background-color: var(--comfy-menu-bg);
+    position: fixed;
+    overflow:hidden;
+    top: 50%;
+    left: 50%;
+    transform: translate(-50%, -50%);
+    z-index: 9999;
+    border-radius: 4px;
+    box-shadow: 4px 4px 14px rgba(255,255,255,0.2);
+  `
+
+  // Create modal header
+  const headerElement = document.createElement('div')
+  headerElement.classList.add('modal-header')
+  headerElement.style.cssText = `
+      display: flex;
+      padding: 20px 24px 8px 24px;
+      justify-content: space-between;
+    `
+
+  const headTitleElement = document.createElement('a')
+  headTitleElement.classList.add('header-title')
+  headTitleElement.style.cssText = `
+      color: var(--descrip-text);
+      font-size: 18px;
+      display: flex;
+      align-items: center;
+      flex: 1;
+      overflow: hidden;
+      text-decoration: none;
+      font-weight: bold;
+      justify-content: space-between;
+      padding: 20px;
+      cursor: pointer;
+      user-select: none;
+    `
+
+  // headTitleElement.href = 'https://github.com/shadowcz007/comfyui-mixlab-nodes'
+  // headTitleElement.target = '_blank'
+  const linkIcon = document.createElement('small')
+  linkIcon.textContent = showTextByLanguage('Auto Open', {
+    'Auto Open': '自动开启'
+  })
+  linkIcon.style.padding = '4px'
+
+  const n_gpu = document.createElement('input')
+  n_gpu.type = 'number'
+  n_gpu.setAttribute('min', -1)
+  n_gpu.setAttribute('max', 9999)
+
+  n_gpu.style = `color: var(--input-text);
+  background-color: var(--comfy-input-bg);
+  border-radius: 8px;
+  border-color: var(--border-color);
+  height: 26px;
+  padding: 4px 10px;
+  width: 48px;
+  margin-left: 12px;`
+  if (localStorage.getItem('_mixlab_llama_n_gpu')) {
+    n_gpu.value = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
+  } else {
+    n_gpu.value = -1
+    localStorage.setItem('_mixlab_llama_n_gpu', -1)
+  }
+
+  const n_gpu_p = document.createElement('p')
+  n_gpu_p.innerText = 'n_gpu_layers'
+
+  const n_gpu_div = document.createElement('div')
+  n_gpu_div.style = `display: flex;
+  justify-content: center;
+  align-items: center;
+  font-size: 12px;`
+  n_gpu_div.appendChild(n_gpu_p)
+  n_gpu_div.appendChild(n_gpu)
+
+  const title = document.createElement('p')
+  title.innerText = 'Models'
+  title.style = `font-size: 18px;
+  margin-right: 8px;`
+
+  const left_d = document.createElement('div')
+  left_d.style = `display: flex;
+  justify-content: center;
+  align-items: center;
+  font-size: 12px;`
+  left_d.appendChild(title)
+
+  left_d.appendChild(linkIcon)
+
+  headTitleElement.appendChild(left_d)
+
+  headTitleElement.appendChild(n_gpu_div)
+
+  //重启
+  const reStart = document.createElement('small')
+  reStart.textContent = showTextByLanguage('restart', {
+    restart: '重启'
+  })
+
+  reStart.style.padding = '4px'
+
+  headTitleElement.appendChild(reStart)
+
+  if (localStorage.getItem('_mixlab_auto_llama_open')) {
+    linkIcon.style.backgroundColor = '#66ff6c'
+    linkIcon.style.color = 'black'
+  }
+  linkIcon.addEventListener('click', e => {
+    e.stopPropagation()
+    if (localStorage.getItem('_mixlab_auto_llama_open')) {
+      localStorage.setItem('_mixlab_auto_llama_open', '')
+      linkIcon.style.backgroundColor = ''
+      linkIcon.style.color = 'var(--descrip-text)'
+    } else {
+      localStorage.setItem('_mixlab_auto_llama_open', 'true')
+      linkIcon.style.backgroundColor = '#66ff6c'
+      linkIcon.style.color = 'black'
+    }
+  })
+
+  reStart.addEventListener('click', e => {
+    e.stopPropagation()
+    div.remove()
+    fetch('mixlab/re_start', {
+      method: 'POST'
+    })
+  })
+
+  n_gpu.addEventListener('click', e => {
+    e.stopPropagation()
+    localStorage.setItem('_mixlab_llama_n_gpu', n_gpu.value)
+  })
+
+  modal.appendChild(headTitleElement)
+
+  // Create modal content area
+  var modalContent = document.createElement('div')
+  modalContent.classList.add('modal-content')
+
+  var input = document.createElement('textarea')
+  input.className = 'comfy-multiline-input'
+  input.style = `    height: 260px;
+  width: 480px;
+  font-size: 16px;
+  padding: 18px;`
+  input.value = localStorage.getItem('_mixlab_system_prompt')
+
+  input.addEventListener('change', e => {
+    e.stopPropagation()
+    localStorage.setItem('_mixlab_system_prompt', input.value)
+  })
+
+  input.addEventListener('click', e => {
+    e.stopPropagation()
+  })
+
+  modalContent.appendChild(input)
+
+  for (const m of models) {
+    let d = document.createElement('div')
+    d.innerText = m
+    d.className = `mix_tag`
+
+    if (!window._mixlab_llamacpp) {
+      d.addEventListener('click', async e => {
+        e.stopPropagation()
+        div.remove()
+        let res = await start_llama(m)
+        window._mixlab_llamacpp = res
+
+        localStorage.setItem('_mixlab_llama_select', res.model)
+
+        if (document.body.querySelector('#mixlab_chatbot_by_llamacpp')) {
+          document.body
+            .querySelector('#mixlab_chatbot_by_llamacpp')
+            .setAttribute('title', window._mixlab_llamacpp.url)
+        }
+      })
+    }
+
+    modalContent.appendChild(d)
+  }
+  modal.appendChild(modalContent)
+
+  const helpInfo = document.createElement('a')
+  helpInfo.textContent = showTextByLanguage('Help', {
+    Help: '寻求帮助'
+  })
+  helpInfo.style = `text-align: center;
+  display: block;
+  padding: 8px;
+  cursor: pointer;
+  font-size: 12px;
+  color: white;`
+  helpInfo.href="https://discord.gg/cXs9vZSqeK"
+  helpInfo.target="_blank"
+  modal.appendChild(helpInfo)
+
+  document.body.appendChild(div)
+}
+
 function createModal (url, markdown, title) {
  // Create modal element
  var div =
    document.querySelector('#mix-modal') || document.createElement('div')
  div.id = 'mix-modal'
  div.innerHTML = ''
-  div.style.cssText = `width: 100%;
+  div.style.cssText = `
+    width: 100%;
    z-index: 9990;
    height: 100vh;
    display: flex;
@@ -895,9 +1312,46 @@ function drawBadge (node, orig, restArgs) {
  return r
 }

+function convertImageUrlToBase64 (imageUrl) {
+  return fetch(imageUrl)
+    .then(response => response.blob())
+    .then(blob => {
+      return new Promise((resolve, reject) => {
+        const reader = new FileReader()
+        reader.onloadend = () => resolve(reader.result)
+        reader.onerror = reject
+        reader.readAsDataURL(blob)
+      })
+    })
+}
+
+async function getSelectImageNode () {
+  var nodes = app.canvas.selected_nodes
+  let imageNode = null
+  if (Object.keys(app.canvas.selected_nodes).length == 0) return
+  for (var id in nodes) {
+    if (nodes[id].imgs) {
+      let base64 = await convertImageUrlToBase64(nodes[id].imgs[0].currentSrc)
+      imageNode = await resizeImage(base64)
+    }
+  }
+  return imageNode
+}
+
 app.registerExtension({
  name: 'Comfy.Mixlab.ui',
  init () {
+    //是否要自动加载模型
+    if (localStorage.getItem('_mixlab_auto_llama_open')) {
+      let model = localStorage.getItem('_mixlab_llama_select')
+      start_llama(model).then(res => {
+        window._mixlab_llamacpp = res
+        document.body
+          .querySelector('#mixlab_chatbot_by_llamacpp')
+          .setAttribute('title', res.url)
+      })
+    }
+
    LGraphCanvas.prototype.helpAboutNode = async function (node) {
      nodesMap =
        nodesMap && Object.keys(nodesMap).length > 0
@@ -922,75 +1376,169 @@ app.registerExtension({

    smart_init()

-    const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions // store the existing method
-    LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
-      // replace it
-      const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
-      node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
-      console.log('getNodeMenuOptions', node.type == 'CLIPTextEncode')
+    LGraphCanvas.prototype.text2text = async function (node) {
+      // console.log(node)
+      let widget = node.widgets.filter(
+        w => w.name === 'text' && typeof w.value == 'string'
+      )[0]
+      if (widget) {
+        app.canvas.centerOnNode(node)

-      let opts = [
-        {
-          content: 'Help ♾️Mixlab', // with a name
-          callback: () => {
-            LGraphCanvas.prototype.helpAboutNode(node)
-          } // and the callback
-        },
-        {
-          content: 'Fix node v2', // with a name
-          callback: () => {
-            LGraphCanvas.prototype.fixTheNode(node)
+        let controller = new AbortController()
+        let ends = [] //TODO 判断终止 <|im_start|>
+        let userInput = widget.value
+        widget.value = widget.value.trim()
+        widget.value += '\n'
+        try {
+          await completion_(
+            window._mixlab_llamacpp.url + '/v1/chat/completions',
+            [
+              {
+                role: 'system',
+                content: localStorage.getItem('_mixlab_system_prompt')
+              },
+              { role: 'user', content: userInput }
+            ],
+            controller,
+            t => {
+              // console.log(t)
+              widget.value += t
+            }
+          )
+        } catch (error) {
+          //是否要自动加载模型
+          if (localStorage.getItem('_mixlab_auto_llama_open')) {
+            let model = localStorage.getItem('_mixlab_llama_select')
+            start_llama(model).then(async res => {
+              window._mixlab_llamacpp = res
+              document.body
+                .querySelector('#mixlab_chatbot_by_llamacpp')
+                .setAttribute('title', res.url)
+
+              await completion_(
+                window._mixlab_llamacpp.url + '/v1/chat/completions',
+                [
+                  {
+                    role: 'system',
+                    content: localStorage.getItem('_mixlab_system_prompt')
+                  },
+                  { role: 'user', content: userInput }
+                ],
+                controller,
+                t => {
+                  // console.log(t)
+                  widget.value += t
+                }
+              )
+            })
          }
        }
-      ]

-      if (node.widgets) {
-        // let text_widget = node.widgets.filter(
-        //   w => w.name === 'text' && typeof w.value == 'string'
-        // )
-        // if (text_widget && text_widget.length == 1) {
-        //   opts.push({
-        //     content: 'Text-to-Text ♾️Mixlab', // with a name
-        //     callback: () => {
-        //       LGraphCanvas.prototype.text2text(node)
-        //     } // and the callback
-        //   })
-        // }
+        widget.value = widget.value.trim()
      }
+    }

-      opts = addSmartMenu(opts, node)
+    LGraphCanvas.prototype.image2text = async function (node) {
+      let imageBase64 = await getSelectImageNode()

-      // if (node.type == 'CLIPTextEncode') {
-      //   // 则出现 randomPrompt
-      //   // CLIPTextEncode 的widget ，name== 'text'
-      //   let node_widget_name = 'text'
-      //   const widget = node.widgets.filter(w => w.name === node_widget_name)[0]
+      if (imageBase64) {
+        // console.log('image2text')
+        // 添加note 节点
+        const NoteNode = LiteGraph.createNode('Note')
+        NoteNode.title = `Image-to-Text ${node.id}`
+        NoteNode.size = [NoteNode.size[0] + 100, NoteNode.size[1]]
+        let widget = NoteNode.widgets[0]
+        widget.value = ''

-      //  let mixlab_nodes_smart_connect= [{node_type:'CLIPTextEncode',
-      //   node_widget_name:'text',
-      //   inputNodeName:'RandomPrompt',
-      //   inputNode_output_type:'STRING'}]
+        NoteNode.pos = [node.pos[0] + node.size[0] + 24, node.pos[1] - 48]

-      //   if (widget) {
-      //     opts = [
-      //       {
-      //         content: 'RandomPrompt',
-      //         callback: () => {
-      //           LGraphCanvas.prototype._createNodeForInput(
-      //             node, //当前node
-      //             widget,//当前node里需要自动连线的widget
-      //             'RandomPrompt',//作为input的node type
-      //             'STRING'// 作为input的node的outputs的type. the input slot type of the target node
-      //           )
-      //         }
-      //       },
-      //       null,
-      //       ...opts
-      //     ]
-      //   }
-      // }
+        app.canvas.graph.add(NoteNode, false)
+        app.canvas.centerOnNode(NoteNode)

-      return [...opts, null, ...options] // and return the options
+        let controller = new AbortController()
+        let ends = []
+        let userInput = widget.value
+        widget.value = widget.value.trim()
+        widget.value += '\n'
+
+        try {
+          await completion_(
+            window._mixlab_llamacpp.url + '/v1/chat/completions',
+            [
+              {
+                role: 'system',
+                content: localStorage.getItem('_mixlab_system_prompt')
+              },
+              // { role: 'user', content: userInput }
+
+              {
+                role: 'user',
+                content: [
+                  {
+                    type: 'image_url',
+                    image_url: {
+                      url: imageBase64
+                    }
+                  },
+                  { type: 'text', text: 'What’s in this image?' }
+                ]
+              }
+            ],
+            controller,
+            t => {
+              // console.log(t)
+              widget.value += t
+
+              NoteNode.size[1] = widget.element.scrollHeight + 20
+              widget.computedHeight = NoteNode.size[1]
+              app.canvas.centerOnNode(NoteNode)
+            }
+          )
+        } catch (error) {
+          //是否要自动加载模型
+          if (localStorage.getItem('_mixlab_auto_llama_open')) {
+            let model = localStorage.getItem('_mixlab_llama_select')
+            start_llama(model).then(async res => {
+              window._mixlab_llamacpp = res
+              document.body
+                .querySelector('#mixlab_chatbot_by_llamacpp')
+                .setAttribute('title', res.url)
+
+              await completion_(
+                window._mixlab_llamacpp.url + '/v1/chat/completions',
+                [
+                  {
+                    role: 'system',
+                    content: localStorage.getItem('_mixlab_system_prompt')
+                  },
+                  {
+                    role: 'user',
+                    content: [
+                      {
+                        type: 'image_url',
+                        image_url: {
+                          url: imageBase64
+                        }
+                      },
+                      { type: 'text', text: 'What’s in this image?' }
+                    ]
+                  }
+                ],
+                controller,
+                t => {
+                  // console.log(t)
+                  widget.value += t
+                  NoteNode.size[1] = widget.element.scrollHeight + 20
+                  widget.computedHeight = NoteNode.size[1]
+                  app.canvas.centerOnNode(NoteNode)
+                }
+              )
+            })
+          }
+        }
+
+        widget.value = widget.value.trim()
+      }
    }

    const getGroupMenuOptions = LGraphCanvas.prototype.getGroupMenuOptions // store the existing method
@@ -1139,6 +1687,70 @@ app.registerExtension({
      this.setDirty(true, true)
    }

+    const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions
+    LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
+      // replace it
+      const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
+      node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
+
+      let opts = [
+        {
+          content: 'Help ♾️Mixlab', // with a name
+          callback: () => {
+            LGraphCanvas.prototype.helpAboutNode(node)
+          } // and the callback
+        },
+        {
+          content: 'Fix node v2', // with a name
+          callback: () => {
+            LGraphCanvas.prototype.fixTheNode(node)
+          }
+        }
+      ]
+
+      if (node.widgets) {
+        let text_widget = node.widgets.filter(
+          w => w.name === 'text' && typeof w.value == 'string'
+        )
+
+        let text_input = node.inputs?.filter(
+          inp => inp.name == 'text' && inp.type == 'STRING'
+        )
+
+        if (
+          text_input &&
+          text_input.length == 0 &&
+          text_widget &&
+          text_widget.length == 1 &&
+          window._mixlab_llamacpp &&
+          node.type != 'ShowTextForGPT'
+        ) {
+          opts.push({
+            content: 'Text-to-Text ♾️Mixlab', // with a name
+            callback: () => {
+              LGraphCanvas.prototype.text2text(node)
+            } // and the callback
+          })
+        }
+
+        if (
+          node.imgs &&
+          node.imgs.length > 0 &&
+          window._mixlab_llamacpp &&
+          window._mixlab_llamacpp.chat_format === 'llava-1-5'
+        ) {
+          opts.push({
+            content: 'Image-to-Text ♾️Mixlab', // with a name
+            callback: () => {
+              LGraphCanvas.prototype.image2text(node)
+            } // and the callback
+          })
+        }
+      }
+
+      return [...opts, null, ...options] // and return the options
+    }
+
    // 支持app模式的json
    const loadAppJson = async data => {
      let workflow
@@ -1173,8 +1785,6 @@ app.registerExtension({
        event.preventDefault()
        event.stopPropagation()

-       
-
        // Dragging from Chrome->Firefox there is a file but its a bmp, so ignore that
        if (
          event.dataTransfer.files.length &&
@@ -1182,13 +1792,14 @@ app.registerExtension({
        ) {
          const reader = new FileReader()
          reader.onload = async () => {
-       
            loadAppJson(reader.result)
          }
          reader.readAsText(event.dataTransfer.files[0])
        }
      })
    }
+
+    createMenu()
  },
  setup () {
    setTimeout(async () => {
@@ -1198,7 +1809,7 @@ app.registerExtension({
      const apps = await get_my_app()
      if (!apps) return

-      console.log('apps',apps)
+      console.log('apps', apps)

      let apps_map = { 0: [] }

@@ -0,0 +1,347 @@
+/* juxtapose - v1.2.2 - 2020-09-03
+ * Copyright (c) 2020 Alex Duner and Northwestern University Knight Lab
+ */
+div.juxtapose {
+	width: 100%;
+	font-family: Helvetica, Arial, sans-serif;
+}
+
+div.jx-slider {
+	width: 100%;
+	height: 100%;
+	position: relative;
+	overflow: hidden;
+	cursor: pointer;
+	color: #f3f3f3;
+}
+
+
+div.jx-handle {
+	position: absolute;
+	height: 100%;
+	width: 40px;
+	cursor: col-resize;
+	z-index: 15;
+	margin-left: -20px;
+}
+
+.vertical div.jx-handle {
+	height: 40px;
+	width: 100%;
+	cursor: row-resize;
+	margin-top: -20px;
+	margin-left: 0;
+}
+
+div.jx-control {
+	height: 100%;
+	margin-right: auto;
+	margin-left: auto;
+	width: 3px;
+	background-color: currentColor;
+}
+
+.vertical div.jx-control {
+	height: 3px;
+	width: 100%;
+	background-color: currentColor;
+	position: relative;
+	top: 50%;
+	transform: translateY(-50%);
+}
+
+div.jx-controller {
+	position: absolute;
+	margin: auto;
+	top: 0;
+	bottom: 0;
+	height: 60px;
+	width: 9px;
+	margin-left: -3px;
+	background-color: currentColor;
+}
+
+.vertical div.jx-controller {
+	height: 9px;
+	width: 100px;
+	margin-left: auto;
+	margin-right: auto;
+	top: -3px;
+	position: relative;
+}
+
+div.jx-arrow {
+	position: absolute;
+	margin: auto;
+	top: 0;
+	bottom: 0;
+	width: 0;
+	height: 0;
+	transition: all .2s ease;
+}
+
+.vertical div.jx-arrow {
+	position: absolute;
+	margin: 0 auto;
+	left: 0;
+	right: 0;
+	width: 0;
+	height: 0;
+	transition: all .2s ease;
+}
+
+
+div.jx-arrow.jx-left {
+	left: 2px;
+	border-style: solid;
+	border-width: 8px 8px 8px 0;
+	border-color: transparent currentColor transparent transparent;
+}
+
+div.jx-arrow.jx-right {
+	right: 2px;
+	border-style: solid;
+	border-width: 8px 0 8px 8px;
+	border-color: transparent transparent transparent currentColor;
+}
+
+.vertical div.jx-arrow.jx-left {
+	left: 0px;
+	top: 2px;
+	border-style: solid;
+	border-width: 0px 8px 8px 8px;
+	border-color: transparent transparent currentColor transparent;
+}
+
+.vertical div.jx-arrow.jx-right {
+	right: 0px;
+	top: auto;
+	bottom: 2px;
+	border-style: solid;
+	border-width: 8px 8px 0 8px;
+	border-color: currentColor transparent transparent transparent;
+}
+
+div.jx-handle:hover div.jx-arrow.jx-left,
+div.jx-handle:active div.jx-arrow.jx-left {
+	left: -1px;
+}
+
+div.jx-handle:hover div.jx-arrow.jx-right,
+div.jx-handle:active div.jx-arrow.jx-right {
+	right: -1px;
+}
+
+.vertical div.jx-handle:hover div.jx-arrow.jx-left,
+.vertical div.jx-handle:active div.jx-arrow.jx-left {
+	left: 0px;
+	top: 0px;
+}
+
+.vertical div.jx-handle:hover div.jx-arrow.jx-right,
+.vertical div.jx-handle:active div.jx-arrow.jx-right {
+	right: 0px;
+	bottom: 0px;
+}
+
+
+div.jx-image {
+	position: absolute;
+	height: 100%;
+	display: inline-block;
+	top: 0;
+	overflow: hidden;
+	-webkit-backface-visibility: hidden;
+}
+
+.vertical div.jx-image {
+	width: 100%;
+	left: 0;
+	top: auto;
+}
+
+div.jx-image img {
+	height: 100%;
+	width: auto;
+	z-index: 5;
+	position: absolute;
+	margin-bottom: 0;
+
+	max-height: none;
+	max-width: none;
+	max-height: initial;
+	max-width: initial;
+}
+
+.vertical div.jx-image img {
+	height: auto;
+	width: 100%;
+}
+
+div.jx-image.jx-left {
+	left: 0;
+	background-position: left;
+}
+
+div.jx-image.jx-left img {
+	left: 0;
+}
+
+div.jx-image.jx-right {
+	right: 0;
+	background-position: right;
+}
+
+div.jx-image.jx-right img {
+	right: 0;
+	bottom: 0;
+}
+
+
+.veritcal div.jx-image.jx-left {
+	top: 0;
+	background-position: top;
+}
+
+.veritcal div.jx-image.jx-left img {
+	top: 0;
+}
+
+.vertical div.jx-image.jx-right {
+	bottom: 0;
+	background-position: bottom;
+}
+
+.veritcal div.jx-image.jx-right img {
+	bottom: 0;
+}
+
+
+div.jx-image div.jx-label {
+	font-size: 1em;
+	padding: .25em .75em;
+	position: relative;
+	display: inline-block;
+	top: 0;
+	background-color: #000; /* IE 8 */
+	background-color: rgba(0,0,0,.7);
+	color: white;
+	z-index: 10;
+	white-space: nowrap;
+	line-height: 18px;
+	vertical-align: middle;
+}
+
+div.jx-image.jx-left div.jx-label {
+	float: left;
+	left: 0;
+}
+
+div.jx-image.jx-right div.jx-label {
+	float: right;
+	right: 0;
+}
+
+.vertical div.jx-image div.jx-label {
+	display: table;
+	position: absolute;
+}
+
+.vertical div.jx-image.jx-right div.jx-label {
+	left: 0;
+	bottom: 0;
+	top: auto;
+}
+
+div.jx-credit {
+	line-height: 1.1;
+	font-size: 0.75em;
+}
+
+div.jx-credit em {
+	font-weight: bold;
+	font-style: normal;
+}
+
+
+/* Animation */
+
+div.jx-image.transition {
+	transition: width .5s ease;
+}
+
+div.jx-handle.transition {
+	transition: left .5s ease;
+}
+
+.vertical div.jx-image.transition {
+	transition: height .5s ease;
+}
+
+.vertical div.jx-handle.transition {
+	transition: top .5s ease;
+}
+
+/* Knight Lab Credit */
+a.jx-knightlab {
+	background-color: #000; /* IE 8 */
+	background-color: rgba(0,0,0,.25);
+	bottom: 0;
+	display: table;
+	height: 14px;
+	line-height: 14px;
+	padding: 1px 4px 1px 5px;
+	position: absolute;
+	right: 0;
+	text-decoration: none;
+	z-index: 10;
+}
+
+a.jx-knightlab div.knightlab-logo {
+	display: inline-block;
+	vertical-align: middle;
+	height: 8px;
+	width: 8px;
+	background-color: #c34528;
+	transform: rotate(45deg);
+	-ms-transform: rotate(45deg);
+	-webkit-transform: rotate(45deg);
+	top: -1.25px;
+	position: relative;
+	cursor: pointer;
+}
+
+a.jx-knightlab:hover {
+	background-color: #000; /* IE 8 */
+	background-color: rgba(0,0,0,.35);
+}
+a.jx-knightlab:hover div.knightlab-logo {
+	background-color: #ce4d28;
+}
+
+a.jx-knightlab span.juxtapose-name {
+	display: table-cell;
+	margin: 0;
+	padding: 0;
+	font-family: Helvetica, Arial, sans-serif;
+	font-weight: 300;
+	color: white;
+	font-size: 10px;
+	padding-left: 0.375em;
+	vertical-align: middle;
+	line-height: normal;
+	text-shadow: none;
+}
+
+/* keyboard accessibility */
+div.jx-controller:focus,
+div.jx-image.jx-left div.jx-label:focus,
+div.jx-image.jx-right div.jx-label:focus,
+a.jx-knightlab:focus {
+	background: #eae34a;
+	color: #000;
+}
+a.jx-knightlab:focus span.juxtapose-name{
+	color: #000;
+	border: none;
+}
Author	SHA1	Message	Date
shadowcz007	6cb9df700b	增加ComparingTwoFrames、右键image-to-text	2024-05-14 09:59:39 +08:00
shadowcz007	4fdda537b9	Update README.md	2024-05-14 09:57:51 +08:00
shadowcz007	cb6f32465a	Update README.md	2024-05-14 09:55:34 +08:00
shadowcz007	c235e36cb4	Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes	2024-05-14 09:54:27 +08:00
shadowcz007	aaca440a94	Update README.md	2024-05-14 09:54:24 +08:00
shadow	1f57950a29	Update README.md	2024-05-14 09:39:57 +08:00
shadow	3d8855ec72	Update README.md	2024-05-14 09:39:45 +08:00
shadowcz007	ac9231d9f3	Update image_mixlab.js	2024-05-13 21:43:20 +08:00
shadowcz007	37c0a56d89	add ComparingTwoFrames	2024-05-13 21:33:44 +08:00
shadowcz007	010915dac4	add help	2024-05-13 11:37:17 +08:00
shadowcz007	83a8b3b970	Update ui_mixlab.js	2024-05-13 10:58:36 +08:00
shadowcz007	f130202aa1	resizeImage	2024-05-12 22:25:08 +08:00
shadowcz007	c8f6800bcd	add image-to-text :llava-phi-3-mini-gguf	2024-05-12 17:34:53 +08:00
shadowcz007	078f9f5dd4	Update ui_mixlab.js	2024-05-12 00:03:11 +08:00
shadowcz007	c0de178c7d	add re_start	2024-05-11 23:58:05 +08:00
shadowcz007	1c767b538d	set n_gpu_layers	2024-05-11 17:53:18 +08:00
shadowcz007	51aab44b5d	Update ui_mixlab.js	2024-05-11 14:43:35 +08:00
shadowcz007	b8a0d4a67b	Update ui_mixlab.js	2024-05-11 14:17:01 +08:00
shadowcz007	cd0dcfbb8c	v0.25.1	2024-05-11 14:12:00 +08:00
shadow	fcc9e30eae	Update __init__.py	2024-05-11 12:55:14 +08:00
shadowcz007	5f66218a43	修复 sys.stdout.isatty() object has no attribute 'isatty'	2024-05-11 12:28:14 +08:00
shadowcz007	61ef4f9a0f	Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes	2024-05-11 12:13:34 +08:00
shadowcz007	f0a8734b42	修复 sys.stdout.isatty() object has no attribute 'isatty'	2024-05-11 12:13:31 +08:00
shadow	4fe95ef4ec	Update README.md	2024-05-11 08:53:33 +08:00
shadowcz007	2a5148845b	yaml	2024-05-10 14:38:32 +08:00
shadowcz007	8fa562caaf	Update install.bat	2024-05-09 09:55:34 +08:00
shadowcz007	cab5620cd5	Update index.html	2024-05-08 23:42:48 +08:00
shadowcz007	be38d36677	Update index.html	2024-05-08 23:32:20 +08:00
shadowcz007	69236fca89	Update __init__.py	2024-05-08 22:55:42 +08:00
shadowcz007	4a4f376bfd	Update __init__.py	2024-05-08 22:53:39 +08:00
shadowcz007	fd9718fe24	Update __init__.py	2024-05-08 22:50:30 +08:00
shadowcz007	26a6e11212	llama_cpp	2024-05-08 22:42:34 +08:00
shadowcz007	de1a669f6e	Update README.md	2024-05-08 10:11:16 +08:00
shadowcz007	e482c9e5c4	0.25.0 text-to-text for prompt	2024-05-07 21:26:47 +08:00
shadowcz007	4e96a77a41	Update README.md	2024-05-07 21:12:27 +08:00
shadowcz007	3346290e5c	Update ImageNode.py	2024-05-07 20:59:22 +08:00
shadowcz007	dcac593efe	Update install.bat	2024-05-07 12:49:03 +08:00
shadowcz007	7248d0de02	update	2024-05-07 12:14:42 +08:00
shadowcz007	8ad3ce632c	Update index.html	2024-05-07 00:08:35 +08:00
shadowcz007	9398b02562	Update ImageNode.py	2024-05-06 22:13:09 +08:00
shadowcz007	164e4da99d	Update ImageNode.py	2024-05-05 18:04:00 +08:00
shadowcz007	0025ea6119	output defaultImage	2024-05-05 13:16:14 +08:00
shadowcz007	eef53a5165	Update ImageNode.py	2024-05-05 12:59:21 +08:00
shadowcz007	9ea066d948	composite_images add position	2024-05-05 12:20:57 +08:00
shadowcz007	4bd900c4a1	Update __init__.py	2024-05-04 09:44:01 +08:00
shadowcz007	736cd2bebd	兼容旧版comfyui	2024-05-04 09:37:25 +08:00
shadowcz007	cd658c2a60	Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes	2024-05-03 22:08:43 +08:00
shadowcz007	9730658f21	Update __init__.py	2024-05-03 22:08:40 +08:00
shadow	3fa107acb1	Update ui_mixlab.js	2024-05-03 18:35:39 +08:00
shadowcz007	7da22179a0	text-to-text	2024-05-03 00:30:16 +08:00
shadowcz007	fc7b71ee78	Update ui_mixlab.js	2024-05-02 19:02:33 +08:00
shadowcz007	b79573bf1f	Update __init__.py	2024-05-02 19:02:28 +08:00
shadowcz007	a01db6f7c0	Update README.md	2024-05-02 12:09:47 +08:00