Compare commits

..
62 Commits
Author SHA1 Message Date
shadowcz007 6cb9df700b 增加ComparingTwoFrames、右键image-to-text 2024-05-14 09:59:39 +08:00
shadowcz007 4fdda537b9 Update README.md 2024-05-14 09:57:51 +08:00
shadowcz007 cb6f32465a Update README.md 2024-05-14 09:55:34 +08:00
shadowcz007 c235e36cb4 Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes 2024-05-14 09:54:27 +08:00
shadowcz007 aaca440a94 Update README.md 2024-05-14 09:54:24 +08:00
shadow 1f57950a29 Update README.md 2024-05-14 09:39:57 +08:00
shadow 3d8855ec72 Update README.md 2024-05-14 09:39:45 +08:00
shadowcz007 ac9231d9f3 Update image_mixlab.js 2024-05-13 21:43:20 +08:00
shadowcz007 37c0a56d89 add ComparingTwoFrames 2024-05-13 21:33:44 +08:00
shadowcz007 010915dac4 add help 2024-05-13 11:37:17 +08:00
shadowcz007 83a8b3b970 Update ui_mixlab.js 2024-05-13 10:58:36 +08:00
shadowcz007 f130202aa1 resizeImage 2024-05-12 22:25:08 +08:00
shadowcz007 c8f6800bcd add image-to-text :llava-phi-3-mini-gguf 2024-05-12 17:34:53 +08:00
shadowcz007 078f9f5dd4 Update ui_mixlab.js 2024-05-12 00:03:11 +08:00
shadowcz007 c0de178c7d add re_start 2024-05-11 23:58:05 +08:00
shadowcz007 1c767b538d set n_gpu_layers 2024-05-11 17:53:18 +08:00
shadowcz007 51aab44b5d Update ui_mixlab.js 2024-05-11 14:43:35 +08:00
shadowcz007 b8a0d4a67b Update ui_mixlab.js 2024-05-11 14:17:01 +08:00
shadowcz007 cd0dcfbb8c v0.25.1 2024-05-11 14:12:00 +08:00
shadow fcc9e30eae Update __init__.py 2024-05-11 12:55:14 +08:00
shadowcz007 5f66218a43 修复 sys.stdout.isatty() object has no attribute 'isatty' 2024-05-11 12:28:14 +08:00
shadowcz007 61ef4f9a0f Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes 2024-05-11 12:13:34 +08:00
shadowcz007 f0a8734b42 修复 sys.stdout.isatty() object has no attribute 'isatty' 2024-05-11 12:13:31 +08:00
shadow 4fe95ef4ec Update README.md 2024-05-11 08:53:33 +08:00
shadowcz007 2a5148845b yaml 2024-05-10 14:38:32 +08:00
shadowcz007 8fa562caaf Update install.bat 2024-05-09 09:55:34 +08:00
shadowcz007 cab5620cd5 Update index.html 2024-05-08 23:42:48 +08:00
shadowcz007 be38d36677 Update index.html 2024-05-08 23:32:20 +08:00
shadowcz007 69236fca89 Update __init__.py 2024-05-08 22:55:42 +08:00
shadowcz007 4a4f376bfd Update __init__.py 2024-05-08 22:53:39 +08:00
shadowcz007 fd9718fe24 Update __init__.py 2024-05-08 22:50:30 +08:00
shadowcz007 26a6e11212 llama_cpp 2024-05-08 22:42:34 +08:00
shadowcz007 de1a669f6e Update README.md 2024-05-08 10:11:16 +08:00
shadowcz007 e482c9e5c4 0.25.0 text-to-text for prompt 2024-05-07 21:26:47 +08:00
shadowcz007 4e96a77a41 Update README.md 2024-05-07 21:12:27 +08:00
shadowcz007 3346290e5c Update ImageNode.py 2024-05-07 20:59:22 +08:00
shadowcz007 dcac593efe Update install.bat 2024-05-07 12:49:03 +08:00
shadowcz007 7248d0de02 update 2024-05-07 12:14:42 +08:00
shadowcz007 8ad3ce632c Update index.html 2024-05-07 00:08:35 +08:00
shadowcz007 9398b02562 Update ImageNode.py 2024-05-06 22:13:09 +08:00
shadowcz007 164e4da99d Update ImageNode.py 2024-05-05 18:04:00 +08:00
shadowcz007 0025ea6119 output defaultImage 2024-05-05 13:16:14 +08:00
shadowcz007 eef53a5165 Update ImageNode.py 2024-05-05 12:59:21 +08:00
shadowcz007 9ea066d948 composite_images add position 2024-05-05 12:20:57 +08:00
shadowcz007 4bd900c4a1 Update __init__.py 2024-05-04 09:44:01 +08:00
shadowcz007 736cd2bebd 兼容旧版comfyui 2024-05-04 09:37:25 +08:00
shadowcz007 cd658c2a60 Merge branch 'main' of https://github.com/shadowcz007/comfyui-mixlab-nodes 2024-05-03 22:08:43 +08:00
shadowcz007 9730658f21 Update __init__.py 2024-05-03 22:08:40 +08:00
shadow 3fa107acb1 Update ui_mixlab.js 2024-05-03 18:35:39 +08:00
shadowcz007 7da22179a0 text-to-text 2024-05-03 00:30:16 +08:00
shadowcz007 fc7b71ee78 Update ui_mixlab.js 2024-05-02 19:02:33 +08:00
shadowcz007 b79573bf1f Update __init__.py 2024-05-02 19:02:28 +08:00
shadowcz007 a01db6f7c0 Update README.md 2024-05-02 12:09:47 +08:00
shadowcz007 117d58c58e v0.24.0 2024-05-02 12:06:19 +08:00
shadowcz007 ff6626ed89 add llama.cpp & Local LLM -Phi-3 & llama3
Phi-3
llama3
2024-05-02 12:02:47 +08:00
shadowcz007 10c798a440 Update index.html 2024-05-02 10:54:20 +08:00
shadowcz007 9097d87819 Update index.html 2024-05-02 10:46:09 +08:00
shadowcz007 d901f503d1 fixbug 2024-05-02 10:06:23 +08:00
shadowcz007 65f2b6ce6f update https 2024-05-02 09:07:28 +08:00
shadowcz007 d949fe8bf1 Update __init__.py 2024-05-01 22:26:46 +08:00
shadowcz007 15cfb48550 Update __init__.py 2024-05-01 20:52:50 +08:00
shadowcz007 447dc6d4c3 fixbug 2024-05-01 15:55:43 +08:00
19 changed files with 2054 additions and 258 deletions
+98 -64
View File
@@ -1,9 +1,26 @@
> 适配了最新版comfyui的py3.11 ,torch 2.1.2+cu121
![](https://img.shields.io/github/release/shadowcz007/comfyui-mixlab-nodes)
> 适配了最新版 comfyui 的 py3.11 ,torch 2.1.2+cu121
> [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)
####
[comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab)
##### `最新`:
ChatGPT 节点支持 Local LLM(llama.cpp),Phi3、llama3 都可以直接一个节点运行了。模型下载后,放置到 `models/llamafile/`
- 右键菜单支持 text-to-text,方便对 prompt 词补全
强烈推荐:[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main),备选:[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
- 右键菜单支持 image-to-text,使用多模态模型,多模态使用 [llava-phi-3-mini-gguf](https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main),注意需要把llava-phi-3-mini-mmproj-f16.gguf也下载
![](./assets/prompt_ai_setup.png)
![](./assets/prompt-ai.png)
#### `相关插件推荐`
<!-- [comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab) -->
[comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)
@@ -13,12 +30,12 @@
<!-- [comfyui-CLIPSeg](https://github.com/shadowcz007/comfyui-CLIPSeg) -->
## 🚀🚗🚚🏃 Workflow-to-APP
## 🚀🚗🚚🏃 Workflow-to-APP
- 新增AppInfo节点,可以通过简单的配置,把workflow转变为一个Web APP。
- 支持多个web app 切换
- 发布为app的workflow,可以在右键里再次编辑了
- web app可以设置分类,在comfyui右键菜单可以编辑更新web app
- 新增 AppInfo 节点,可以通过简单的配置,把 workflow 转变为一个 Web APP。
- 支持多个 web app 切换
- 发布为 app 的 workflow,可以在右键里再次编辑了
- web app 可以设置分类,在 comfyui 右键菜单可以编辑更新 web app
- 支持动态提示
![](./assets/微信图片_20240421205440.png)
@@ -28,7 +45,6 @@
- The workflow, which is now released as an app, can also be edited again by right-clicking.
- The web app can be configured with categories, and the web app can be edited and updated in the right-click menu of ComfyUI.
![](./assets/0-m-app.png)
![](./assets/appinfo-readme.png)
@@ -36,59 +52,91 @@
![](./assets/appinfo-2.png)
Example:
- workflow
![APP info](./workflow/appinfo-workflow.svg)
[text-to-image](./workflow/Text-to-Image-app.json)
![APP info](./workflow/appinfo-workflow.svg)
[text-to-image](./workflow/Text-to-Image-app.json)
APP-JSON:
- [text-to-image](./example/Text-to-Image_3.json)
- [image-to-image](./example/Image-to-Image_2.json)
- text-to-text
> 暂时支持 9 种节点作为界面上的输入节点:Load Image、VHS_LoadVideo、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
> 暂时支持 9 种节点作为界面上的输入节点:Load Image、VHS*LoadVideo、CLIPTextEncode、PromptSlide、TextInput*、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
> 输出节点:PreviewImage 、SaveImage、ShowTextForGPT、VHS_VideoCombine、PromptImage
> seed统一输入控件,支持:SamplerCustom、KSampler
> seed 统一输入控件,支持:SamplerCustom、KSampler
> 配套[ps插件](https://github.com/shadowcz007/comfyui-ps-plugin)
> 配套[ps 插件](https://github.com/shadowcz007/comfyui-ps-plugin)
> 如果遇到上传图片不成功,请检查下:局域网或者是云服务,请使用https,端口8189这个服务( 感谢 @Damien 反馈问题)
> 如果遇到上传图片不成功,请检查下:局域网或者是云服务,请使用 https,端口 8189 这个服务( 感谢 @Damien 反馈问题)
> If you encounter difficulties in uploading images, please check the following: for local network or cloud services, please use HTTPS and the service on port 8189. (Thanks to @Damien for reporting the issue.)
## 🏃🚗🚚🚀 Real-time Design
## 🏃🚗🚚🚀 Real-time Design
> ScreenShareNode & FloatingVideoNode. Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
![screenshare](./assets/screenshare.png)
https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43e-410a-ab3a-1952b7b4e7da
<!-- [ScreenShareNode](./workflow/2-screeshare.json) -->
[ScreenShareNode & FloatingVideoNode](./workflow/3-FloatVideo-workflow.json)
!! Please use the address with HTTPS (https://127.0.0.1).
### SpeechRecognition & SpeechSynthesis
![f](./assets/audio-workflow.svg)
[Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
### GPT
> Support for calling multiple GPTs.ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
> Support for calling multiple GPTs.Local LLM(llama.cpp)、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
![gpt-workflow.svg](./assets/gpt-workflow.svg)
[workflow-5](./workflow/5-gpt-workflow.json)
最新:ChatGPT 节点支持 Local LLM(llama.cpp),Phi3、llama3 都可以直接一个节点运行了。
Model download,move to :`models/llamafile/`
强烈推荐:[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)
备选:[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
> 如果碰到安装失败,可以尝试手动安装
```
../../../python_embeded/python.exe -s -m pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
../../../python_embeded/python.exe -s -m pip install llama-cpp-python[server]
```
> [Mac](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)
```
pip uninstall llama-cpp-python -y
CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --no-cache-dir
pip install 'llama-cpp-python[server]'
```
```
pip install llama-cpp-python \
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
```
## Prompt
> PromptSlide
![](./assets/prompt_weight.png)
> ![](./assets/prompt_weight.png)
<!-- ![](./workflow/promptslide-appinfo-workflow.svg) -->
@@ -102,20 +150,20 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
> PromptImage & PromptSimplification,Assist in simplifying prompt words, comparing images and prompt word nodes.
> ChinesePrompt && PromptGenerate,中文prompt节点,直接用中文书写你的prompt
> ChinesePrompt && PromptGenerate,中文 prompt 节点,直接用中文书写你的 prompt
![](./assets/ChinesePrompt_workflow.svg)
### Layers
> A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.
![layers](./assets/layers-workflow.svg)
![poster](./assets/poster-workflow.svg)
### 3D
![](./assets/3d-workflow.png)
![](./assets/3d_app.png)
[workflow](./assets/Image-to-3D_1.json)
@@ -123,44 +171,44 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
![](./assets/3dimage.png)
[workflow](./workflow/3D-workflow.json)
### Image
#### LoadImagesToBatch
> Upload multiple images for batch input into the IP adapter.
> Upload multiple images for batch input into the IP adapter.
#### LoadImagesFromLocal
> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.
> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.
![watch](./assets/4-loadfromlocal-watcher-workflow.svg)
[workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)
#### LoadImagesFromURL
> Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.
### Style
> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
![](./assets/VisualStylePrompting.png)
> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
### Utils
> The Color node provides a color picker for easy color selection, the Font node offers built-in font selection for use with TextImage to generate text images, and the DynamicDelayByText node allows delayed execution based on the length of the input text.
- [添加了DynamicDelayByText功能,可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
- [添加了 DynamicDelayByText 功能,可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
- [Added DynamicDelayByText, enabling delayed execution based on input text length.](./workflow/audio-chatgpt-workflow.json)
- [使用CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)
- [使用 CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)
- [CkptNames compare the effects of different models.](./workflow/ckpts-image-workflow.json)
### Other Nodes
![main](./assets/all-workflow.svg)
@@ -168,33 +216,27 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
[workflow-1](./workflow/1-workflow.json)
> TransparentImage
![TransparentImage](./assets/TransparentImage.png)
> FeatheredMask、SmoothMask
Add edges to an image.
![FeatheredMask](./assets/FlVou_Y6kaGWYoEj1Tn0aTd4AjMI.jpg)
> LaMaInpainting
from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainting)
> rembgNode
"briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"
*** briarmbg *** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
**_ briarmbg _** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
### Improvement
### Improvement
- Add "help" option to the context menu for each node.
- Add "Nodes Map" option to the global context menu.
@@ -205,23 +247,21 @@ An improvement has been made to directly redirect to GitHub to search for missin
![node-not-found](./assets/node-not-found.png)
### Models
* [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in ```models/triposr```
- [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in `models/triposr`
* [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main) and place it in ```models/triposr/facebook/dino-vitb16```
- [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main) and place it in `models/triposr/facebook/dino-vitb16`
[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:`models/rembg`
[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:```models/rembg```
[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : `models/lama`
[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : ```models/lama```
[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :`models/clip_interrogator/Salesforce/blip-image-captioning-base`
[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :```models/clip_interrogator/Salesforce/blip-image-captioning-base```
[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:`models/prompt_generator/text2image-prompt-generator`
[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:```models/prompt_generator/text2image-prompt-generator```
[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:```models/prompt_generator/opus-mt-zh-en```
[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:`models/prompt_generator/opus-mt-zh-en`
## Installation
@@ -237,40 +277,35 @@ git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
Install the requirements:
run directly:
```
cd ComfyUI/custom_nodes/comfyui-mixlab-nodes
install.bat
```
or install the requirements using:
```
../../../python_embeded/python.exe -s -m pip install -r requirements.txt
```
If you are using a venv, make sure you have it activated before installation and use:
```
pip3 install -r requirements.txt
```
#### Chinese community
访问 [www.mixcomfy.com](https://www.mixcomfy.com),获得更多内测功能,关注微信公众号:Mixlab无界社区
访问 [www.mixcomfy.com](https://www.mixcomfy.com),获得更多内测功能,关注微信公众号:Mixlab 无界社区
####
####
File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
#### discussions:
[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
<picture>
<source
@@ -290,4 +325,3 @@ File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
src="https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date"
/>
</picture>
+231 -44
View File
@@ -7,9 +7,28 @@ import urllib
import hashlib
import datetime
import folder_paths
import logging
from comfy.cli_args import args
python = sys.executable
#修复 sys.stdout.isatty() object has no attribute 'isatty'
try:
sys.stdout.isatty()
except:
print('#fix sys.stdout.isatty')
sys.stdout.isatty = lambda: False
llama_port=None
llama_model=""
llama_chat_format=""
try:
from .nodes.ChatGPT import get_llama_models,get_llama_model_path,llama_cpp_client
llama_cpp_client("")
except:
print("##nodes.ChatGPT ImportError")
from server import PromptServer
@@ -42,7 +61,7 @@ def is_installed(package, package_overwrite=None):
print(f"Couldn't install\nCommand: {command}\nError code: {result.returncode}")
else:
print(package+'## OK')
try:
import OpenSSL
except ImportError:
@@ -414,37 +433,86 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
runner = web.AppRunner(self.app, access_log=None)
await runner.setup()
if not await check_port_available(address, port):
raise RuntimeError(f"Port {port} is already in use.")
# if not await check_port_available(address, port):
# raise RuntimeError(f"Port {port} is already in use.")
http_success = False
http_port=port
for i in range(11): # 尝试最多11次
if await check_port_available(address, port + i):
http_port = port + i
site = web.TCPSite(runner, address, http_port)
await site.start()
http_success = True
break
site = web.TCPSite(runner, address, port)
await site.start()
if not http_success:
raise RuntimeError(f"Ports {port} to {port + 10} are all in use.")
# site = web.TCPSite(runner, address, port)
# await site.start()
ssl_context = None
scheme = "http"
try:
# 跟着本体修改
if args.tls_keyfile and args.tls_certfile:
scheme = "https"
ssl_context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_SERVER, verify_mode=ssl.CERT_NONE)
ssl_context.load_cert_chain(certfile=args.tls_certfile,
keyfile=args.tls_keyfile)
else:
# 如果没传,则自动创建
import ssl
crt, key = create_for_https()
ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_context.load_cert_chain(crt, key)
except:
import ssl
crt, key = create_for_https()
ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_context.load_cert_chain(crt, key)
import ssl
crt, key = create_for_https()
ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_context.load_cert_chain(crt, key)
success = False
for i in range(10): # 尝试最多10次
if await check_port_available(address, port + 1 + i):
https_port = port + 1 + i
for i in range(11): # 尝试最多11次
if await check_port_available(address, http_port + 1 + i):
https_port = http_port + 1 + i
site2 = web.TCPSite(runner, address, https_port, ssl_context=ssl_context)
await site2.start()
success = True
break
if not success:
raise RuntimeError(f"Ports {port + 1} to {port + 10} are all in use.")
raise RuntimeError(f"Ports {http_port + 1} to {http_port + 10} are all in use.")
if address == '':
address = '0.0.0.0'
address = '127.0.0.1'
if address=='0.0.0.0':
address = '127.0.0.1'
if verbose:
print("\033[93mStarting server\n")
print("\033[93mTo see the GUI go to: http://{}:{}".format(address, port))
print("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))
logging.info("\n")
logging.info("\n\nStarting server")
# print("\033[93mStarting server\n")
logging.info("\033[93mTo see the GUI go to: http://{}:{}".format(address, http_port))
logging.info("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))
# print("\033[93mTo see the GUI go to: http://{}:{}".format(address, http_port))
# print("\033[93mTo see the GUI go to: https://{}:{}\033[0m".format(address, https_port))
if call_on_start is not None:
call_on_start(address, port)
try:
if scheme=='https':
call_on_start(scheme,address, https_port)
else:
call_on_start(scheme,address, http_port)
except:
call_on_start(address,http_port)
except Exception as e:
print(f"Error starting the server: {e}")
@@ -455,7 +523,6 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
# webbrowser.open(f"https://{address}")
# webbrowser.open(f"http://{address}:{port}")
PromptServer.start=new_start
# 创建路由表
@@ -550,13 +617,19 @@ async def nodes_map_hander(request):
async def get_checkpoints(request):
data = await request.json()
t="checkpoints"
names=[]
try:
t=data['type']
names = folder_paths.get_filename_list(t)
except Exception as e:
print('/mixlab/folder_paths',False,e)
names = folder_paths.get_filename_list(t)
try:
if data['type']=='llamafile':
names=get_llama_models()
except:
print("llamafile none")
return web.json_response({"names":names,"types":list(folder_paths.folder_names_and_paths.keys())})
@@ -577,20 +650,130 @@ async def post_prompt_result(request):
return web.json_response({"result":res})
# 扩展api接口
# from server import PromptServer
# from aiohttp import web
# @routes.post('/ws_image')
# async def my_hander_method(request):
# post = await request.post()
# x = post.get("something")
# return web.json_response({})
async def start_local_llm(data):
global llama_port,llama_model,llama_chat_format
if llama_port and llama_model and llama_chat_format:
return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
import threading
import uvicorn
from llama_cpp.server.app import create_app
from llama_cpp.server.settings import (
Settings,
ServerSettings,
ModelSettings,
ConfigFileSettings,
)
if not "model" in data and "model_path" in data:
data['model']= os.path.basename(data["model_path"])
model=data["model_path"]
elif "model" in data:
model=get_llama_model_path(data['model'])
n_gpu_layers=-1
if "n_gpu_layers" in data:
n_gpu_layers=data['n_gpu_layers']
chat_format="chatml"
model_alias=os.path.basename(model)
# 多模态
clip_model_path=None
prefix = "llava-phi-3-mini"
file_name = prefix+"-mmproj-"
if model_alias.startswith(prefix):
for file in os.listdir(os.path.dirname(model)):
if file.startswith(file_name):
clip_model_path=os.path.join(os.path.dirname(model),file)
chat_format='llava-1-5'
print('#clip_model_path',chat_format,clip_model_path)
address="127.0.0.1"
port=9090
success = False
for i in range(11): # 尝试最多11次
if await check_port_available(address, port + i):
port = port + i
success = True
break
if success == False:
return {"port":None,"model":""}
server_settings=ServerSettings(host=address,port=port)
app = create_app(
server_settings=server_settings,
model_settings=[
ModelSettings(
model=model,
model_alias=os.path.basename(model),
n_gpu_layers=n_gpu_layers,
n_ctx=4098,
chat_format=chat_format,
embedding=False,
clip_model_path=clip_model_path
)])
def run_uvicorn():
uvicorn.run(
app,
host=os.getenv("HOST", server_settings.host),
port=int(os.getenv("PORT", server_settings.port)),
ssl_keyfile=server_settings.ssl_keyfile,
ssl_certfile=server_settings.ssl_certfile,
)
# 创建一个子线程
thread = threading.Thread(target=run_uvicorn)
# 启动子线程
thread.start()
llama_port=port
llama_model=data['model']
llama_chat_format=chat_format
return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
# llam服务的开启
@routes.post('/mixlab/start_llama')
async def my_hander_method(request):
data =await request.json()
# print(data)
try:
result=await start_local_llm(data)
except:
result={
{"port":None,"model":"","llama_cpp_error":True}
}
print('start_local_llm error')
return web.json_response(result)
# 重启服务
@routes.post('/mixlab/re_start')
def re_start(request):
try:
sys.stdout.close_log()
except Exception as e:
pass
return os.execv(sys.executable, [sys.executable] + sys.argv)
# 导入节点
from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
from .nodes.ImageNode import LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
from .nodes.ImageNode import ComparingTwoFrames,LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
# from .nodes.Vae import VAELoader,VAEDecode
from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
@@ -651,6 +834,7 @@ NODE_CLASS_MAPPINGS = {
# "VAELoaderConsistencyDecoder":VAELoader,
"SaveImageToLocal":SaveImageToLocal,
"SaveImageAndMetadata_":SaveImageAndMetadata,
"ComparingTwoFrames_":ComparingTwoFrames,
# "VAEDecodeConsistencyDecoder":VAEDecode,
"ScreenShare":ScreenShareNode,
"FloatingVideo":FloatingVideo,
@@ -705,6 +889,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
"IntNumber":"Int Input ♾️MixlabApp",
"ImagesPrompt_":"Images Input ♾️MixlabApp",
"SaveImageAndMetadata_":"Save Image Output ♾️MixlabApp",
"ComparingTwoFrames_":"Comparing Two Frames ♾️MixlabApp",
"ResizeImageMixlab":"Resize Image ♾️Mixlab",
"RandomPrompt": "Random Prompt ♾️Mixlab",
"PromptImage":"Output Prompt and Image ♾️Mixlab",
@@ -713,7 +898,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
"VAEDecodeConsistencyDecoder":"Consistency Decoder Decode",
"ScreenShare":"Screen Share ♾️Mixlab",
"FloatingVideo":"FloatingVideo ♾️Mixlab",
"ChatGPTOpenAI":"ChatGPT ♾️Mixlab",
"ChatGPTOpenAI":"ChatGPT & Local LLM ♾️Mixlab",
"ShowTextForGPT":"Show Text ♾️MixlabApp",
"MergeLayers":"Merge Layers ♾️Mixlab",
"SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
@@ -756,42 +941,44 @@ NODE_DISPLAY_NAME_MAPPINGS = {
# web ui的节点功能
WEB_DIRECTORY = "./web"
print('--------------')
print('\033[91m ### Mixlab Nodes: \033[93mLoaded')
logging.info('--------------')
logging.info('\033[91m ### Mixlab Nodes: \033[93mLoaded')
# print('\033[91m ### Mixlab Nodes: \033[93mLoaded')
try:
from .nodes.Lama import LaMaInpainting
print('LaMaInpainting.available',LaMaInpainting.available)
logging.info('LaMaInpainting.available {}'.format(LaMaInpainting.available))
if LaMaInpainting.available:
NODE_CLASS_MAPPINGS['LaMaInpainting']=LaMaInpainting
except Exception as e:
print('LaMaInpainting.available',False,e)
logging.info('LaMaInpainting.available False')
try:
from .nodes.ClipInterrogator import ClipInterrogator
print('ClipInterrogator.available',ClipInterrogator.available)
logging.info('ClipInterrogator.available {}'.format(ClipInterrogator.available))
if ClipInterrogator.available:
NODE_CLASS_MAPPINGS['ClipInterrogator']=ClipInterrogator
except Exception as e:
print('ClipInterrogator.available',False,e)
logging.info('ClipInterrogator.available False')
try:
from .nodes.TextGenerateNode import PromptGenerate,ChinesePrompt
print('PromptGenerate.available',PromptGenerate.available)
logging.info('PromptGenerate.available {}'.format(PromptGenerate.available))
if PromptGenerate.available:
NODE_CLASS_MAPPINGS['PromptGenerate_Mix']=PromptGenerate
print('ChinesePrompt.available',ChinesePrompt.available)
logging.info('ChinesePrompt.available {}'.format(ChinesePrompt.available))
if ChinesePrompt.available:
NODE_CLASS_MAPPINGS['ChinesePrompt_Mix']=ChinesePrompt
except Exception as e:
print('TextGenerateNode.available',False,e)
logging.info('TextGenerateNode.available False')
try:
from .nodes.RembgNode import RembgNode_
print('RembgNode_.available',RembgNode_.available)
logging.info('RembgNode_.available {}'.format(RembgNode_.available))
if RembgNode_.available:
NODE_CLASS_MAPPINGS['RembgNode_Mix']=RembgNode_
except Exception as e:
print('RembgNode_.available',False,e)
logging.info('RembgNode_.available False' )
print('\033[93m -------------- \033[0m')
logging.info('\033[93m -------------- \033[0m')
Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

+6
View File
@@ -10,6 +10,12 @@ if exist "%python_exec%" (
for /f "delims=" %%i in (%requirements_txt%) do (
%python_exec% -s -m pip install "%%i" -i https://pypi.tuna.tsinghua.edu.cn/simple
)
%python_exec% -s -m pip install --upgrade --force llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
%python_exec% -s -m pip install --upgrade --force llama-cpp-python[server]
) else (
echo Installing with system Python
for /f "delims=" %%i in (%requirements_txt%) do (
+109 -17
View File
@@ -87,16 +87,102 @@ def ZhipuAI_client(key):
return client
# 优先使用phi
def phi_sort(lst):
return sorted(lst, key=lambda x: x.lower().count('phi'), reverse=True)
def get_llama_path():
try:
return folder_paths.get_folder_paths('llamafile')[0]
except:
return os.path.join(folder_paths.models_dir, "llamafile")
def get_llama_models():
res=[]
model_path=get_llama_path()
if os.path.exists(model_path):
files = os.listdir(model_path)
for file in files:
if os.path.isfile(os.path.join(model_path, file)):
res.append(file)
res=phi_sort(res)
return res
llama_modes_list=get_llama_models()
def get_llama_model_path(file_name):
model_path=get_llama_path()
mp=os.path.join(model_path,file_name)
return mp
def llama_cpp_client(file_name):
try:
if is_installed('llama_cpp')==False:
import subprocess
# 安装
print('#pip install llama-cpp-python')
result = subprocess.run([sys.executable, '-s', '-m', 'pip',
'install',
'llama-cpp-python',
'--extra-index-url',
'https://abetlen.github.io/llama-cpp-python/whl/cu121'
], capture_output=True, text=True)
#检查命令执行结果
if result.returncode == 0:
print("#install success")
from llama_cpp import Llama
subprocess.run([sys.executable, '-s', '-m', 'pip',
'install',
'llama-cpp-python[server]'
], capture_output=True, text=True)
else:
print("#install error")
else:
from llama_cpp import Llama
except:
print("#install llama-cpp-python error")
if file_name:
mp=get_llama_model_path(file_name)
# file_name=get_llama_models()[0]
# model_path=os.path.join(folder_paths.models_dir, "llamafile")
# mp=os.path.join(model_path,file_name)
llm = Llama(model_path=mp, chat_format="chatml",n_gpu_layers=-1,n_ctx=512)
return llm
def chat(client, model_name,messages ):
try_count = 0
while True:
try_count += 1
try:
response = client.chat.completions.create(
model=model_name,
messages=messages
)
if hasattr(client, "chat"):
response = client.chat.completions.create(
model=model_name,
messages=messages
)
else:
# 是llama的
response = client.create_chat_completion_openai_v1(
messages=messages,
# response_format={
# "type": "json_object",
# },
# temperature=0.7,
)
break
except openai.AuthenticationError as ex:
raise ex
@@ -105,7 +191,8 @@ def chat(client, model_name,messages ):
raise ex
time.sleep(3)
continue
# print(response.keys())
finish_reason = response.choices[0].finish_reason
if finish_reason != "stop":
raise RuntimeError("API finished with unexpected reason: " + finish_reason)
@@ -128,6 +215,16 @@ class ChatGPTNode:
@classmethod
def INPUT_TYPES(cls):
model_list=llama_modes_list+[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-35-turbo",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-4-0613",
"gpt-4-1106-preview",
"glm-4"
]
return {
"required": {
"api_key":("KEY", {"default": "", "multiline": True,"dynamicPrompts": False}),
@@ -138,16 +235,8 @@ class ChatGPTNode:
"default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
"multiline": True,"dynamicPrompts": False
}),
"model": ([
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-35-turbo",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-4-0613",
"gpt-4-1106-preview",
"glm-4"],
{"default": "gpt-3.5-turbo"}),
"model": ( model_list,
{"default": model_list[0]}),
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "step": 1}),
"context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
},
@@ -170,8 +259,8 @@ class ChatGPTNode:
api_url,
prompt,
system_content,
model,
seed,context_size,unique_id = None, extra_pnginfo=None):
model,
seed,context_size,unique_id = None, extra_pnginfo=None):
# print(api_key!='',api_url,prompt,system_content,model,seed)
# 可以选择保留会话历史以维持上下文记忆
# 或者在此处清除会话历史 self.session_history.clear()
@@ -193,6 +282,9 @@ class ChatGPTNode:
if model == "glm-4" :
client = ZhipuAI_client(api_key) # 使用 Zhipuai 的接口
print('using Zhipuai interface')
elif model in llama_modes_list:
#
client=llama_cpp_client(model)
else :
client = openai_client(api_key,api_url) # 使用 ChatGPT 的接口
print('using ChatGPT interface')
+9 -3
View File
@@ -70,14 +70,20 @@ def load_caption_model(model_path,config,t='blip-base'):
return (caption_model,caption_processor)
def get_clip_interrogator_path():
try:
return folder_paths.get_folder_paths('clip_interrogator')[0]
except:
return os.path.join(folder_paths.models_dir, "clip_interrogator")
caption_model_path=os.path.join(folder_paths.models_dir, "clip_interrogator/Salesforce/blip-image-captioning-base")
cache_path=get_clip_interrogator_path()
caption_model_path=os.path.join(cache_path, "Salesforce/blip-image-captioning-base")
if not os.path.exists(caption_model_path):
print(f"## clip_interrogator_model not found: {caption_model_path}, pls download from https://huggingface.co/Salesforce/blip-image-captioning-base")
caption_model_path='Salesforce/blip-image-captioning-base'
cache_path=os.path.join(folder_paths.models_dir, "clip_interrogator")
# Tensor to PIL
def tensor2pil(image):
View File
+248 -37
View File
@@ -29,23 +29,122 @@ def opencv_to_pil(image):
return pil_image
def composite_images(foreground, background, mask):
def composite_images(foreground, background, mask,is_multiply_blend=False,position="overall"):
width,height=foreground.size
bg_image=background
bwidth,bheight=bg_image.size
# 按z-index排序
layer = {
"x":0,
"y":0,
"width":width,
"height":height,
"z_index":88,
"scale_option":'overall',
"image":foreground,
"mask":mask
}
if position=="overall":
layer = {
"x":0,
"y":0,
"width":bwidth,
"height":bheight,
"z_index":88,
"scale_option":'overall',
"image":foreground,
"mask":mask
}
elif position=='center_bottom':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":int(bwidth*0.75*0.5),
"y":bheight-new_height-24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
elif position=='right_bottom':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":bwidth-int(bwidth*0.25)-24,
"y":bheight-new_height-24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
elif position=='center_top':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":int( bwidth*0.75*0.5),
"y":24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
elif position=='right_top':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":bwidth-int(bwidth*0.25)-24,
"y":24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
elif position=='left_top':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":24,
"y":24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
elif position=='left_bottom':
scale = int(bwidth*0.25) / width
new_height = int(height * scale)
layer = {
"x":24,
"y":bheight-new_height-24,
"width":int(bwidth*0.25),
"height":int(bheight*0.25),
"z_index":88,
"scale_option":'width',
"image":foreground,
"mask":mask
}
width, height = bg_image.size
# width, height = bg_image.size
layer_image=layer['image']
layer_mask=layer['mask']
@@ -53,12 +152,12 @@ def composite_images(foreground, background, mask):
bg_image=merge_images(bg_image,
layer_image,
layer_mask,
layer['x'],
layer['y'],
layer['width'],
layer['height'],
layer['scale_option']
)
layer['x'],
layer['y'],
layer['width'],
layer['height'],
layer['scale_option'],
is_multiply_blend )
bg_image=bg_image.convert('RGB')
@@ -704,7 +803,6 @@ def multiply_blend(image1, image2):
# cv2.imwrite('result.jpg', result)
def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,is_multiply_blend=False):
# 打开底图
bg_image = bg_image.convert("RGBA")
@@ -730,21 +828,42 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,
# 整体缩放
layer_image = layer_image.resize((width, height))
elif scale_option == "longest":
original_width, original_height = layer_image.size
if original_width > original_height:
new_width=width
scale = width / original_width
new_height = int(original_height * scale)
x=0
y=int((height-new_height)*0.5)
else:
new_height=height
scale = height / original_height
new_width = int(original_height * scale)
x=int((width-new_width)*0.5)
y=0
# elif side == "shortest":
# if width < height:
#
# else:
#
# 调整mask的大小
nw, nh = layer_image.size
mask = mask.resize((nw, nh))
# 分离出a通道
r, g, b, alpha = layer_image.split()
alpha = ImageOps.invert(alpha)
# 创建一个新的RGB图像
new_rgb_image = Image.new("RGB", layer_image.size)
# 将透明通道粘贴到新的RGB图像上
new_rgb_image.paste(layer_image, (0, 0), mask=alpha)
# # 分离出a通道
# r, g, b, alpha = layer_image.split()
# alpha = ImageOps.invert(alpha)
# # 创建一个新的RGB图像
# new_rgb_image = Image.new("RGB", layer_image.size)
# # 将透明通道粘贴到新的RGB图像上
# new_rgb_image.paste(layer_image, (0, 0), mask=alpha)
new_rgb_image.paste(layer_image, (x, y), mask=mask)
mask=new_rgb_image.convert('L')
mask = ImageOps.invert(mask)
# new_rgb_image.paste(layer_image, (x, y), mask=mask)
# mask=new_rgb_image.convert('L')
# mask = ImageOps.invert(mask)
if is_multiply_blend:
bg_image_white=Image.new("RGB", bg_image.size,(255, 255, 255))
@@ -753,8 +872,11 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,
bg_image=multiply_blend(bg_image_white,bg_image)
bg_image=bg_image.convert("RGBA")
else:
# 在底图上粘贴图层
bg_image.paste(layer_image, (x, y), mask=mask)
transparent_img = Image.new("RGBA",layer_image.size, (255, 255, 255, 0))
transparent_img.paste(layer_image,(0, 0), mask)
# transparent_img.save('test.png')
bg_image.paste(transparent_img, (x, y), transparent_img)
# 输出合成后的图片
return bg_image
@@ -802,6 +924,31 @@ def resize_image(layer_image, scale_option, width, height,color="white"):
resized_image = resized_image.convert("RGB")
resized_image=resize_2(resized_image)
return resized_image
elif scale_option == "longest":
#暂时不用,
if original_width > original_height:
new_width=width
scale = width / original_width
new_height = int(original_height * scale)
x=0
y=int((new_height-height)*0.5)
resized_image = Image.new("RGB", (new_width, new_height), color=color)
resized_image.paste(layer_image.resize((new_width, new_height)), (x,y))
resized_image = resized_image.convert("RGB")
resized_image=resize_2(resized_image)
return resized_image
else:
new_height=height
scale = height / original_height
new_width = int(original_height * scale)
x=int((new_width-width)*0.5)
y=0
resized_image = Image.new("RGB", (new_width, new_height), color=color)
resized_image.paste(layer_image.resize((new_width, new_height)), (x,y))
resized_image = resized_image.convert("RGB")
resized_image=resize_2(resized_image)
return resized_image
layer_image=resize_2(layer_image)
return layer_image
@@ -1703,10 +1850,15 @@ class CompositeImages:
def INPUT_TYPES(s):
return {
"required": {
"foreground": ("IMAGE",),
"mask":("MASK",),
"background": ("IMAGE",),
},
"foreground": (any_type,),
"mask":("MASK",),
"background": ("IMAGE",),
},
"optional":{
"is_multiply_blend": ("BOOLEAN", {"default": False}),
"position": (['overall',"center_bottom","center_top","right_bottom","left_bottom","right_top","left_top"],),
}
}
RETURN_TYPES = ("IMAGE",)
@@ -1718,11 +1870,11 @@ class CompositeImages:
# OUTPUT_IS_LIST = (True,)
def run(self, foreground,mask,background):
def run(self, foreground,mask,background,is_multiply_blend,position):
foreground= tensor2pil(foreground)
mask= tensor2pil(mask)
background= tensor2pil(background)
res=composite_images(foreground,background,mask)
res=composite_images(foreground,background,mask,is_multiply_blend,position)
return (pil2tensor(res),)
@@ -2831,6 +2983,65 @@ class SaveImageAndMetadata:
return { "ui": { "images": results } }
class ComparingTwoFrames:
def __init__(self):
self.output_dir = folder_paths.get_output_directory()
self.type = "output"
self.prefix_append = "ComparingTwoFrames"
self.compress_level = 4
@classmethod
def INPUT_TYPES(s):
return {"required":
{"before_image": ("IMAGE", ),
"after_image": ("IMAGE", )
},
}
RETURN_TYPES = ()
FUNCTION = "comparingImages"
OUTPUT_NODE = True
CATEGORY = "♾️Mixlab/Output"
def comparingImages(self, before_image,after_image):
filename_prefix = self.prefix_append
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
filename_prefix, self.output_dir, after_image[0].shape[1], after_image[0].shape[0])
bresults = list()
for bimage in before_image:
i = 255. * bimage.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
file = f"{filename}_{counter:05}_.png"
img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
bresults.append({
"filename": file,
"subfolder": subfolder,
"type": self.type
})
counter += 1
results = list()
for aimage in after_image:
i = 255. * aimage.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
file = f"{filename}_{counter:05}_.png"
img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
results.append({
"filename": file,
"subfolder": subfolder,
"type": self.type
})
counter += 1
return { "ui": { "after_images": results,"before_images":bresults } }
class ImageColorTransfer:
@classmethod
def INPUT_TYPES(s):
+7 -2
View File
@@ -42,8 +42,13 @@ else:
_available=True
llma_model_path=os.path.join(folder_paths.models_dir, "lama/big-lama.pt")
def get_lama_path():
try:
return folder_paths.get_folder_paths('lama')[0]
except:
return os.path.join(folder_paths.models_dir, "lama")
llma_model_path=os.path.join(get_lama_path(), "big-lama.pt")
if not os.path.exists(llma_model_path):
os.environ['LAMA_MODEL']=''
print(f"## lama torchscript model not found: {llma_model_path},pls download from https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt")
+9 -3
View File
@@ -18,19 +18,25 @@ from lark import Lark, Transformer, v_args
global _available
_available=True
def get_text_generator_path():
try:
return folder_paths.get_folder_paths('prompt_generator')[0]
except:
return os.path.join(folder_paths.models_dir, "prompt_generator")
text_generator_model_path=os.path.join(folder_paths.models_dir, "prompt_generator/text2image-prompt-generator")
prompt_generator=get_text_generator_path()
text_generator_model_path=os.path.join(prompt_generator, "text2image-prompt-generator")
if not os.path.exists(text_generator_model_path):
print(f"## text_generator_model not found: {text_generator_model_path}, pls download from https://huggingface.co/succinctly/text2image-prompt-generator/tree/main")
text_generator_model_path='succinctly/text2image-prompt-generator'
zh_en_model_path=os.path.join(folder_paths.models_dir, "prompt_generator/opus-mt-zh-en")
zh_en_model_path=os.path.join(prompt_generator, "opus-mt-zh-en")
if not os.path.exists(zh_en_model_path):
print(f"## zh_en_model not found: {zh_en_model_path}, pls download from https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main")
zh_en_model_path='Helsinki-NLP/opus-mt-zh-en'
def is_installed(package):
try:
spec = importlib.util.find_spec(package)
+23 -13
View File
@@ -247,10 +247,10 @@
.card textarea {
width: 100%;
/* height: 200px; */
height:'fit-content';
/* min-width: 300px; */
margin-top: 12px;
resize: vertical;
resize: none;
overflow: hidden;
}
@@ -404,9 +404,9 @@
display: none;
}
/* 定义滚动条轨道的背景颜色 */
/* 定义滚动条轨道的背景颜色 */
::-webkit-scrollbar-track {
::-webkit-scrollbar-track {
background-color: #f1f1f1;
/* 轨道背景颜色 */
}
@@ -434,14 +434,18 @@
/* 角落颜色 */
}
/*
.dynamic_prompt::after {
content: attr(title);
position: absolute;
position: absolute;
color: black;
padding: 4px;
padding-left: 25px;
border-radius: 4px;
} */
summary {
user-select: none;
}
</style>
<!-- <script src="../../../scripts/api.js" type="module"></script> -->
@@ -948,6 +952,10 @@
function createOutputs(outputData, link) {
const url = new URL(window.location.href);
const params = new URLSearchParams(url.search);
const innerApp = params.get("innerApp");
const container = document.createElement('div');
container.className = "output";
@@ -956,11 +964,11 @@
const copyHTML = document.createElement('button');
copyHTML.innerText = 'copy as html'
action.appendChild(copyHTML)
if (!innerApp) action.appendChild(copyHTML)
const copyImage = document.createElement('button');
copyImage.innerText = 'copy image'
action.appendChild(copyImage)
if (!innerApp) action.appendChild(copyImage)
copyImage.style.marginLeft = '18px';
let isURL = false;
@@ -1062,18 +1070,19 @@
"Image Save",
"SaveImageAndMetadata_",
"TransparentImage"].includes(node.class_type)) {
console.log('output#image', node)
const url = node.options?.defaultImage || window._appData?.icon || base64Df;
let a = document.createElement('a');
a.id = `output_${node.id}`
a.setAttribute('data-pswp-width', "200");
a.setAttribute('data-pswp-height', "200");
a.setAttribute('target', "_blank");
a.setAttribute('href', base64Df);
a.setAttribute('href', url);
a.setAttribute('title', node.title);
let img = new Image();
// img;
img.src = window._appData?.icon || base64Df;
img.src = url;
a.appendChild(img)
output_card.appendChild(a);
isShowImageFn = true;
@@ -2267,7 +2276,8 @@
// leftDiv.appendChild(des);
leftDiv.appendChild(statusDiv);
leftDiv.appendChild(input1);
mainDiv.appendChild(submitButton);
if (typeof (data.data) == 'object') mainDiv.appendChild(submitButton);
rightDiv.appendChild(output);
@@ -2624,7 +2634,7 @@
ui.title.update(appData.name || 'Mixlab APP');
// 更新应用图标
ui.icon.update(appData.icon || base64Df);
ui.icon.update(appData.icon || appData.output[0]?.options?.defaultImage || base64Df);
ui.des.update(appData.description || '-');
+19 -1
View File
@@ -207,9 +207,27 @@ async function extractInputAndOutputData (
// input.push()
}
if (outputIds.includes(id)) {
let options = {}
//输出的默认图
if (
node.type === 'SaveImageAndMetadata_' &&
app.graph.getNodeById(id).imgs
) {
// SaveImageAndMetadata_的默认图,转为base64
let imgurl = app.graph.getNodeById(id).imgs[0].src
options.defaultImage = await drawImageToCanvas(imgurl, 512)
console.log('#SaveImageAndMetadata_的默认图', options)
}
// let node = app.graph.getNodeById(id)
// output.push()
output[outputIds.indexOf(id)] = { ...data[id], title: node.title, id }
output[outputIds.indexOf(id)] = {
...data[id],
title: node.title,
id,
options
}
}
if (
+106
View File
@@ -0,0 +1,106 @@
async function* completion (url, messages, controller) {
let data = {
model: 'gpt-3.5-turbo-16k',
messages,
temperature: 0.6,
stream: true
}
// if (imageNode) {
// data = { ...data, image_data: [imageNode] }
// }
// let controller = new AbortController()
let response = await fetch(url, {
method: 'POST',
body: JSON.stringify(data),
headers: {
Connection: 'keep-alive',
'Content-Type': 'application/json',
Accept: 'text/event-stream'
},
signal: controller.signal
})
const reader = response.body.getReader()
const decoder = new TextDecoder()
let content = ''
let leftover = '' // Buffer for partially read lines
try {
let cont = true
while (cont) {
let result = await reader.read()
if (result.done) {
break
}
// Add any leftover data to the current chunk of data
const text = leftover + decoder.decode(result.value)
// Check if the last character is a line break
const endsWithLineBreak = text.endsWith('\n')
// Split the text into lines
let lines = text.split('\n')
// If the text doesn't end with a line break, then the last line is incomplete
// Store it in leftover to be added to the next chunk of data
if (!endsWithLineBreak) {
leftover = lines.pop()
} else {
leftover = '' // Reset leftover if we have a line break at the end
}
// Parse all sse events and add them to result
const regex = /^(\S+):\s(.*)$/gm
for (const line of lines) {
const match = regex.exec(line)
if (match) {
result[match[1]] = match[2]
// since we know this is llama.cpp, let's just decode the json in data
if (result.data) {
result.data = JSON.parse(result.data)
// console.log('#result.data',result.data)
content += result.data.choices[0].delta?.content || ''
// yield
yield result
// if we got a stop token from server, we will break here
if (result.data.choices[0].finish_reason == 'stop') {
if (result.data.generation_settings) {
// generation_settings = result.data.generation_settings;
}
cont = false
break
}
}
}
}
}
} catch (e) {
console.error('llama error: ', e)
throw e
} finally {
controller.abort()
}
return content
// return (await response.json()).content
}
export async function completion_ (url, messages, controller, callback) {
let request = await completion(url, messages, controller)
for await (const chunk of request) {
let content = chunk.data.choices[0].delta.content || ''
if (chunk.data.choices[0].role == 'assistant') {
//开始
content = ''
}
if (callback) callback(content)
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
const repoOwner = 'shadowcz007' // 替换为仓库的所有者
const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
const version = 'v0.23.0'
const version = 'v0.26.0'
fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
.then(response => response.json())
+149
View File
@@ -705,7 +705,9 @@ app.registerExtension({
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'LoadImagesToBatch') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = function () {
orig_nodeCreated?.apply(this, arguments)
@@ -841,3 +843,150 @@ app.registerExtension({
}
}
})
// 如何引入css
app.registerExtension({
name: 'Mixlab.output.ComparingTwoFrames_',
init () {
$el('link', {
rel: 'stylesheet',
href: '/extensions/comfyui-mixlab-nodes/lib/juxtapose.css',
parent: document.head
})
$el('style', {
textContent: `
.juxtapose-name{
display: none!important;
}
`,
parent: document.body
})
},
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'ComparingTwoFrames_') {
const onNodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = function () {
const r = onNodeCreated
? onNodeCreated.apply(this, arguments)
: undefined
this.size = [400, this.size[1]]
console.log('##onNodeCreated', this)
const widget = {
type: 'div',
name: 'preview',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, 400, 44, node.size[1])
)
},
serialize: false
}
widget.div = $el('div', {})
document.body.appendChild(widget.div)
this.addCustomWidget(widget)
this.serialize_widgets = true //需要保存参数
const onRemoved = this.onRemoved
this.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
return r
}
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments)
console.log('##onExecuted', this, message)
this.widgets[0].div.id = 'mix_comparingtowframes_' + this.id
let after_image = message.after_images[0]
let before_image = message.before_images[0]
after_image = `${window.location.protocol}//${
window.location.hostname
}:${window.location.port}/view?filename=${encodeURIComponent(
after_image.filename
)}&type=${after_image.type}&subfolder=${encodeURIComponent(
after_image.subfolder
)}&t=${+new Date()}`
before_image = `${window.location.protocol}//${
window.location.hostname
}:${window.location.port}/view?filename=${encodeURIComponent(
before_image.filename
)}&type=${before_image.type}&subfolder=${encodeURIComponent(
before_image.subfolder
)}&t=${+new Date()}`
this.widgets[0].div.innerHTML = ''
let slider = new juxtapose.JXSlider(
'#mix_comparingtowframes_' + this.id,
[
{
src: before_image,
label: 'Before'
},
{
src: after_image,
label: 'After'
}
],
{
animate: true,
showLabels: true,
showCredits: false,
startingPosition: '50%',
makeResponsive: false
}
)
this.widgets_values = [
{
src: before_image,
label: 'Before'
},
{
src: after_image,
label: 'After'
}
]
this.size=[this.size[0],300]
}
}
},
async loadedGraphNode (node, app) {
// console.log('##loadedGraphNode', node)
if (node.type === 'ComparingTwoFrames_') {
// node.widgets[0].div.id = 'mix_comparingtowframes_' + node.id
// if (node.widgets_values && node.widgets_values[0]) {
// node.widgets[0].div.innerHTML = ''
// let slider = new juxtapose.JXSlider(
// '#mix_comparingtowframes_' + node.id,
// node.widgets_values,
// {
// animate: true,
// showLabels: true,
// showCredits: false,
// startingPosition: '50%',
// makeResponsive: false
// }
// )
// }
}
}
})
+684 -73
View File
@@ -9,6 +9,148 @@ import {
import { smart_init, addSmartMenu } from './smart_connect.js'
import { completion_ } from './chat.js'
function showTextByLanguage (key, json) {
// 获取浏览器语言
var language = navigator.language
// 判断是否为中文
if (
language.indexOf('zh') !== -1 ||
(language.indexOf('cn') !== -1 && json[key])
) {
return json[key]
} else {
return key
}
}
//系统prompt
const systemPrompt = `You are a prompt creator, your task is to create prompts for the user input request, the prompts are image descriptions that include keywords for (an adjective, type of image, framing/composition, subject, subject appearance/action, environment, lighting situation, details of the shoot/illustration, visuals aesthetics and artists), brake keywords by comas, provide high quality, non-verboose, coherent, brief, concise, and not superfluous prompts, the subject from the input request must be included verbatim on the prompt,the prompt is english`
if (!localStorage.getItem('_mixlab_system_prompt')) {
localStorage.setItem('_mixlab_system_prompt', systemPrompt)
}
// 获取llama 模型
async function get_llamafile_models () {
try {
const response = await fetch('/mixlab/folder_paths', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
type: 'llamafile'
})
})
const data = await response.json()
console.log(data)
return data.names
} catch (error) {
console.error(error)
}
}
// 运行llama
async function start_llama (model = 'Phi-3-mini-4k-instruct-Q5_K_S.gguf') {
let n_gpu_layers = -1
try {
n_gpu_layers = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
} catch (error) {}
try {
const response = await fetch('/mixlab/start_llama', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
model,
n_gpu_layers
})
})
const data = await response.json()
if (data.llama_cpp_error) {
return
}
return {
url: `http://${window.location.hostname}:${data.port}`,
model: data.model,
chat_format: data.chat_format
}
} catch (error) {
console.error(error)
}
}
function resizeImage (base64Image) {
var img = new Image()
var canvas = document.createElement('canvas')
var ctx = canvas.getContext('2d')
return new Promise((res, rej) => {
img.onload = function () {
// 等比例缩放图片
var width = img.width
var height = img.height
var max_width = 768
if (width > max_width) {
height *= max_width / width
width = max_width
}
// 设置canvas尺寸
canvas.width = width
canvas.height = height
// 在canvas上绘制图片
ctx.drawImage(img, 0, 0, width, height)
// 将canvas转换为base64图片数据
var canvasData = canvas.toDataURL()
res(canvasData) // canvas转换后的base64图片数据
}
img.src = base64Image
})
}
// 菜单入口
async function createMenu () {
const menu = document.querySelector('.comfy-menu')
const separator = document.createElement('div')
separator.style = `margin: 20px 0px;
width: 100%;
height: 1px;
background: var(--border-color);
`
menu.append(separator)
if (!menu.querySelector('#mixlab_chatbot_by_llamacpp')) {
const appsButton = document.createElement('button')
appsButton.id = 'mixlab_chatbot_by_llamacpp'
appsButton.textContent = '♾️Mixlab'
// appsButton.onclick = () =>
appsButton.onclick = async () => {
if (window._mixlab_llamacpp) {
//显示运行的模型
createModelsModal([
window._mixlab_llamacpp.url,
window._mixlab_llamacpp.model
])
} else {
let ms = await get_llamafile_models()
ms = ms.filter(m => !m.match('-mmproj-'))
if (ms.length > 0) createModelsModal(ms)
}
}
menu.append(appsButton)
}
}
let isScriptLoaded = {}
function loadExternalScript (url) {
@@ -299,8 +441,9 @@ async function get_my_app (filename = null, category = '') {
data = []
for (const res of result.data) {
let { app, workflow } = res.data;
if (app?.filename) data.push({
let { app, workflow } = res.data
if (app?.filename)
data.push({
...app,
data: workflow,
date: res.date
@@ -344,6 +487,33 @@ injectCSS(`::-webkit-scrollbar {
width: 2px;
}
#mixlab_chatbot_by_llamacpp{
font-size:14px
}
#mixlab_chatbot_by_llamacpp::before {
content: attr(title);
position: absolute;
margin-top: 24px;
font-size: 10px;
}
.mix_tag{
padding:8px;cursor: pointer;font-size: 14px;
color: var(--input-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;
margin-top: 2px;
margin-bottom: 14px;
}
.mix_tag:hover{
background-color: #101c19;
color: aquamarine;
}
@keyframes loading_mixlab {
0% {
background-color: green;
@@ -494,13 +664,28 @@ app.showMissingNodesError = async function (
// console.log('#nodesMap', nodesMap)
// console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
this.ui.dialog.show(
`When loading the graph, the following node types were not found: <ul>${missingNodeGithub(
missingNodeTypes,
nodesMap
).join('')}</ul>${
hasAddedNodes
? 'Nodes that have failed to load will show as red on the graph.'
: ''
`<a style="color: white;
font-size: 18px;
font-weight: 800;
letter-spacing: 2px;
font-family: sans-serif;
}"
href="https://discord.gg/cXs9vZSqeK" target="_blank">${showTextByLanguage(
'Welcome to Mixlab nodes discord, seeking help.',
{
'Welcome to Mixlab nodes discord, seeking help.':
'寻求帮助,加入Mixlab nodes交流频道'
}
)}</a><br><br>${showTextByLanguage(
'When loading the graph, the following node types were not found:',
{
'When loading the graph, the following node types were not found:':
'缺少以下节点:'
}
)}
<ul>${missingNodeGithub(missingNodeTypes, nodesMap).join('')}</ul>${
hasAddedNodes ? '' : ''
}`
)
this.logging.addEntry('Comfy.App', 'warn', {
@@ -586,13 +771,245 @@ async function fetchReadmeContent (url) {
}
}
function createModelsModal (models) {
var div =
document.querySelector('#model-modal') || document.createElement('div')
div.id = 'model-modal'
div.innerHTML = ''
div.style.cssText = `
width: 100%;
z-index: 9990;
height: 100vh;
display: flex;
color: var(--descrip-text);
position: fixed;
top: 0;
left: 0;
background: #000000a8;
`
var modal = document.createElement('div')
div.addEventListener('click', e => {
e.stopPropagation()
div.remove()
})
div.appendChild(modal)
modal.classList.add('modal-body')
// Set modal styles
modal.style.cssText = `
color: var(--descrip-text);
background-color: var(--comfy-menu-bg);
position: fixed;
overflow:hidden;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
z-index: 9999;
border-radius: 4px;
box-shadow: 4px 4px 14px rgba(255,255,255,0.2);
`
// Create modal header
const headerElement = document.createElement('div')
headerElement.classList.add('modal-header')
headerElement.style.cssText = `
display: flex;
padding: 20px 24px 8px 24px;
justify-content: space-between;
`
const headTitleElement = document.createElement('a')
headTitleElement.classList.add('header-title')
headTitleElement.style.cssText = `
color: var(--descrip-text);
font-size: 18px;
display: flex;
align-items: center;
flex: 1;
overflow: hidden;
text-decoration: none;
font-weight: bold;
justify-content: space-between;
padding: 20px;
cursor: pointer;
user-select: none;
`
// headTitleElement.href = 'https://github.com/shadowcz007/comfyui-mixlab-nodes'
// headTitleElement.target = '_blank'
const linkIcon = document.createElement('small')
linkIcon.textContent = showTextByLanguage('Auto Open', {
'Auto Open': '自动开启'
})
linkIcon.style.padding = '4px'
const n_gpu = document.createElement('input')
n_gpu.type = 'number'
n_gpu.setAttribute('min', -1)
n_gpu.setAttribute('max', 9999)
n_gpu.style = `color: var(--input-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
height: 26px;
padding: 4px 10px;
width: 48px;
margin-left: 12px;`
if (localStorage.getItem('_mixlab_llama_n_gpu')) {
n_gpu.value = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
} else {
n_gpu.value = -1
localStorage.setItem('_mixlab_llama_n_gpu', -1)
}
const n_gpu_p = document.createElement('p')
n_gpu_p.innerText = 'n_gpu_layers'
const n_gpu_div = document.createElement('div')
n_gpu_div.style = `display: flex;
justify-content: center;
align-items: center;
font-size: 12px;`
n_gpu_div.appendChild(n_gpu_p)
n_gpu_div.appendChild(n_gpu)
const title = document.createElement('p')
title.innerText = 'Models'
title.style = `font-size: 18px;
margin-right: 8px;`
const left_d = document.createElement('div')
left_d.style = `display: flex;
justify-content: center;
align-items: center;
font-size: 12px;`
left_d.appendChild(title)
left_d.appendChild(linkIcon)
headTitleElement.appendChild(left_d)
headTitleElement.appendChild(n_gpu_div)
//重启
const reStart = document.createElement('small')
reStart.textContent = showTextByLanguage('restart', {
restart: '重启'
})
reStart.style.padding = '4px'
headTitleElement.appendChild(reStart)
if (localStorage.getItem('_mixlab_auto_llama_open')) {
linkIcon.style.backgroundColor = '#66ff6c'
linkIcon.style.color = 'black'
}
linkIcon.addEventListener('click', e => {
e.stopPropagation()
if (localStorage.getItem('_mixlab_auto_llama_open')) {
localStorage.setItem('_mixlab_auto_llama_open', '')
linkIcon.style.backgroundColor = ''
linkIcon.style.color = 'var(--descrip-text)'
} else {
localStorage.setItem('_mixlab_auto_llama_open', 'true')
linkIcon.style.backgroundColor = '#66ff6c'
linkIcon.style.color = 'black'
}
})
reStart.addEventListener('click', e => {
e.stopPropagation()
div.remove()
fetch('mixlab/re_start', {
method: 'POST'
})
})
n_gpu.addEventListener('click', e => {
e.stopPropagation()
localStorage.setItem('_mixlab_llama_n_gpu', n_gpu.value)
})
modal.appendChild(headTitleElement)
// Create modal content area
var modalContent = document.createElement('div')
modalContent.classList.add('modal-content')
var input = document.createElement('textarea')
input.className = 'comfy-multiline-input'
input.style = ` height: 260px;
width: 480px;
font-size: 16px;
padding: 18px;`
input.value = localStorage.getItem('_mixlab_system_prompt')
input.addEventListener('change', e => {
e.stopPropagation()
localStorage.setItem('_mixlab_system_prompt', input.value)
})
input.addEventListener('click', e => {
e.stopPropagation()
})
modalContent.appendChild(input)
for (const m of models) {
let d = document.createElement('div')
d.innerText = m
d.className = `mix_tag`
if (!window._mixlab_llamacpp) {
d.addEventListener('click', async e => {
e.stopPropagation()
div.remove()
let res = await start_llama(m)
window._mixlab_llamacpp = res
localStorage.setItem('_mixlab_llama_select', res.model)
if (document.body.querySelector('#mixlab_chatbot_by_llamacpp')) {
document.body
.querySelector('#mixlab_chatbot_by_llamacpp')
.setAttribute('title', window._mixlab_llamacpp.url)
}
})
}
modalContent.appendChild(d)
}
modal.appendChild(modalContent)
const helpInfo = document.createElement('a')
helpInfo.textContent = showTextByLanguage('Help', {
Help: '寻求帮助'
})
helpInfo.style = `text-align: center;
display: block;
padding: 8px;
cursor: pointer;
font-size: 12px;
color: white;`
helpInfo.href="https://discord.gg/cXs9vZSqeK"
helpInfo.target="_blank"
modal.appendChild(helpInfo)
document.body.appendChild(div)
}
function createModal (url, markdown, title) {
// Create modal element
var div =
document.querySelector('#mix-modal') || document.createElement('div')
div.id = 'mix-modal'
div.innerHTML = ''
div.style.cssText = `width: 100%;
div.style.cssText = `
width: 100%;
z-index: 9990;
height: 100vh;
display: flex;
@@ -895,9 +1312,46 @@ function drawBadge (node, orig, restArgs) {
return r
}
function convertImageUrlToBase64 (imageUrl) {
return fetch(imageUrl)
.then(response => response.blob())
.then(blob => {
return new Promise((resolve, reject) => {
const reader = new FileReader()
reader.onloadend = () => resolve(reader.result)
reader.onerror = reject
reader.readAsDataURL(blob)
})
})
}
async function getSelectImageNode () {
var nodes = app.canvas.selected_nodes
let imageNode = null
if (Object.keys(app.canvas.selected_nodes).length == 0) return
for (var id in nodes) {
if (nodes[id].imgs) {
let base64 = await convertImageUrlToBase64(nodes[id].imgs[0].currentSrc)
imageNode = await resizeImage(base64)
}
}
return imageNode
}
app.registerExtension({
name: 'Comfy.Mixlab.ui',
init () {
//是否要自动加载模型
if (localStorage.getItem('_mixlab_auto_llama_open')) {
let model = localStorage.getItem('_mixlab_llama_select')
start_llama(model).then(res => {
window._mixlab_llamacpp = res
document.body
.querySelector('#mixlab_chatbot_by_llamacpp')
.setAttribute('title', res.url)
})
}
LGraphCanvas.prototype.helpAboutNode = async function (node) {
nodesMap =
nodesMap && Object.keys(nodesMap).length > 0
@@ -922,75 +1376,169 @@ app.registerExtension({
smart_init()
const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions // store the existing method
LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
// replace it
const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
console.log('getNodeMenuOptions', node.type == 'CLIPTextEncode')
LGraphCanvas.prototype.text2text = async function (node) {
// console.log(node)
let widget = node.widgets.filter(
w => w.name === 'text' && typeof w.value == 'string'
)[0]
if (widget) {
app.canvas.centerOnNode(node)
let opts = [
{
content: 'Help ♾️Mixlab', // with a name
callback: () => {
LGraphCanvas.prototype.helpAboutNode(node)
} // and the callback
},
{
content: 'Fix node v2', // with a name
callback: () => {
LGraphCanvas.prototype.fixTheNode(node)
let controller = new AbortController()
let ends = [] //TODO 判断终止 <|im_start|>
let userInput = widget.value
widget.value = widget.value.trim()
widget.value += '\n'
try {
await completion_(
window._mixlab_llamacpp.url + '/v1/chat/completions',
[
{
role: 'system',
content: localStorage.getItem('_mixlab_system_prompt')
},
{ role: 'user', content: userInput }
],
controller,
t => {
// console.log(t)
widget.value += t
}
)
} catch (error) {
//是否要自动加载模型
if (localStorage.getItem('_mixlab_auto_llama_open')) {
let model = localStorage.getItem('_mixlab_llama_select')
start_llama(model).then(async res => {
window._mixlab_llamacpp = res
document.body
.querySelector('#mixlab_chatbot_by_llamacpp')
.setAttribute('title', res.url)
await completion_(
window._mixlab_llamacpp.url + '/v1/chat/completions',
[
{
role: 'system',
content: localStorage.getItem('_mixlab_system_prompt')
},
{ role: 'user', content: userInput }
],
controller,
t => {
// console.log(t)
widget.value += t
}
)
})
}
}
]
if (node.widgets) {
// let text_widget = node.widgets.filter(
// w => w.name === 'text' && typeof w.value == 'string'
// )
// if (text_widget && text_widget.length == 1) {
// opts.push({
// content: 'Text-to-Text ♾️Mixlab', // with a name
// callback: () => {
// LGraphCanvas.prototype.text2text(node)
// } // and the callback
// })
// }
widget.value = widget.value.trim()
}
}
opts = addSmartMenu(opts, node)
LGraphCanvas.prototype.image2text = async function (node) {
let imageBase64 = await getSelectImageNode()
// if (node.type == 'CLIPTextEncode') {
// // 则出现 randomPrompt
// // CLIPTextEncode 的widget ,name== 'text'
// let node_widget_name = 'text'
// const widget = node.widgets.filter(w => w.name === node_widget_name)[0]
if (imageBase64) {
// console.log('image2text')
// 添加note 节点
const NoteNode = LiteGraph.createNode('Note')
NoteNode.title = `Image-to-Text ${node.id}`
NoteNode.size = [NoteNode.size[0] + 100, NoteNode.size[1]]
let widget = NoteNode.widgets[0]
widget.value = ''
// let mixlab_nodes_smart_connect= [{node_type:'CLIPTextEncode',
// node_widget_name:'text',
// inputNodeName:'RandomPrompt',
// inputNode_output_type:'STRING'}]
NoteNode.pos = [node.pos[0] + node.size[0] + 24, node.pos[1] - 48]
// if (widget) {
// opts = [
// {
// content: 'RandomPrompt',
// callback: () => {
// LGraphCanvas.prototype._createNodeForInput(
// node, //当前node
// widget,//当前node里需要自动连线的widget
// 'RandomPrompt',//作为input的node type
// 'STRING'// 作为input的node的outputs的type. the input slot type of the target node
// )
// }
// },
// null,
// ...opts
// ]
// }
// }
app.canvas.graph.add(NoteNode, false)
app.canvas.centerOnNode(NoteNode)
return [...opts, null, ...options] // and return the options
let controller = new AbortController()
let ends = []
let userInput = widget.value
widget.value = widget.value.trim()
widget.value += '\n'
try {
await completion_(
window._mixlab_llamacpp.url + '/v1/chat/completions',
[
{
role: 'system',
content: localStorage.getItem('_mixlab_system_prompt')
},
// { role: 'user', content: userInput }
{
role: 'user',
content: [
{
type: 'image_url',
image_url: {
url: imageBase64
}
},
{ type: 'text', text: 'What’s in this image?' }
]
}
],
controller,
t => {
// console.log(t)
widget.value += t
NoteNode.size[1] = widget.element.scrollHeight + 20
widget.computedHeight = NoteNode.size[1]
app.canvas.centerOnNode(NoteNode)
}
)
} catch (error) {
//是否要自动加载模型
if (localStorage.getItem('_mixlab_auto_llama_open')) {
let model = localStorage.getItem('_mixlab_llama_select')
start_llama(model).then(async res => {
window._mixlab_llamacpp = res
document.body
.querySelector('#mixlab_chatbot_by_llamacpp')
.setAttribute('title', res.url)
await completion_(
window._mixlab_llamacpp.url + '/v1/chat/completions',
[
{
role: 'system',
content: localStorage.getItem('_mixlab_system_prompt')
},
{
role: 'user',
content: [
{
type: 'image_url',
image_url: {
url: imageBase64
}
},
{ type: 'text', text: 'What’s in this image?' }
]
}
],
controller,
t => {
// console.log(t)
widget.value += t
NoteNode.size[1] = widget.element.scrollHeight + 20
widget.computedHeight = NoteNode.size[1]
app.canvas.centerOnNode(NoteNode)
}
)
})
}
}
widget.value = widget.value.trim()
}
}
const getGroupMenuOptions = LGraphCanvas.prototype.getGroupMenuOptions // store the existing method
@@ -1139,6 +1687,70 @@ app.registerExtension({
this.setDirty(true, true)
}
const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions
LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
// replace it
const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
let opts = [
{
content: 'Help ♾️Mixlab', // with a name
callback: () => {
LGraphCanvas.prototype.helpAboutNode(node)
} // and the callback
},
{
content: 'Fix node v2', // with a name
callback: () => {
LGraphCanvas.prototype.fixTheNode(node)
}
}
]
if (node.widgets) {
let text_widget = node.widgets.filter(
w => w.name === 'text' && typeof w.value == 'string'
)
let text_input = node.inputs?.filter(
inp => inp.name == 'text' && inp.type == 'STRING'
)
if (
text_input &&
text_input.length == 0 &&
text_widget &&
text_widget.length == 1 &&
window._mixlab_llamacpp &&
node.type != 'ShowTextForGPT'
) {
opts.push({
content: 'Text-to-Text ♾️Mixlab', // with a name
callback: () => {
LGraphCanvas.prototype.text2text(node)
} // and the callback
})
}
if (
node.imgs &&
node.imgs.length > 0 &&
window._mixlab_llamacpp &&
window._mixlab_llamacpp.chat_format === 'llava-1-5'
) {
opts.push({
content: 'Image-to-Text ♾️Mixlab', // with a name
callback: () => {
LGraphCanvas.prototype.image2text(node)
} // and the callback
})
}
}
return [...opts, null, ...options] // and return the options
}
// 支持app模式的json
const loadAppJson = async data => {
let workflow
@@ -1173,8 +1785,6 @@ app.registerExtension({
event.preventDefault()
event.stopPropagation()
// Dragging from Chrome->Firefox there is a file but its a bmp, so ignore that
if (
event.dataTransfer.files.length &&
@@ -1182,13 +1792,14 @@ app.registerExtension({
) {
const reader = new FileReader()
reader.onload = async () => {
loadAppJson(reader.result)
}
reader.readAsText(event.dataTransfer.files[0])
}
})
}
createMenu()
},
setup () {
setTimeout(async () => {
@@ -1198,7 +1809,7 @@ app.registerExtension({
const apps = await get_my_app()
if (!apps) return
console.log('apps',apps)
console.log('apps', apps)
let apps_map = { 0: [] }
+347
View File
@@ -0,0 +1,347 @@
/* juxtapose - v1.2.2 - 2020-09-03
* Copyright (c) 2020 Alex Duner and Northwestern University Knight Lab
*/
div.juxtapose {
width: 100%;
font-family: Helvetica, Arial, sans-serif;
}
div.jx-slider {
width: 100%;
height: 100%;
position: relative;
overflow: hidden;
cursor: pointer;
color: #f3f3f3;
}
div.jx-handle {
position: absolute;
height: 100%;
width: 40px;
cursor: col-resize;
z-index: 15;
margin-left: -20px;
}
.vertical div.jx-handle {
height: 40px;
width: 100%;
cursor: row-resize;
margin-top: -20px;
margin-left: 0;
}
div.jx-control {
height: 100%;
margin-right: auto;
margin-left: auto;
width: 3px;
background-color: currentColor;
}
.vertical div.jx-control {
height: 3px;
width: 100%;
background-color: currentColor;
position: relative;
top: 50%;
transform: translateY(-50%);
}
div.jx-controller {
position: absolute;
margin: auto;
top: 0;
bottom: 0;
height: 60px;
width: 9px;
margin-left: -3px;
background-color: currentColor;
}
.vertical div.jx-controller {
height: 9px;
width: 100px;
margin-left: auto;
margin-right: auto;
top: -3px;
position: relative;
}
div.jx-arrow {
position: absolute;
margin: auto;
top: 0;
bottom: 0;
width: 0;
height: 0;
transition: all .2s ease;
}
.vertical div.jx-arrow {
position: absolute;
margin: 0 auto;
left: 0;
right: 0;
width: 0;
height: 0;
transition: all .2s ease;
}
div.jx-arrow.jx-left {
left: 2px;
border-style: solid;
border-width: 8px 8px 8px 0;
border-color: transparent currentColor transparent transparent;
}
div.jx-arrow.jx-right {
right: 2px;
border-style: solid;
border-width: 8px 0 8px 8px;
border-color: transparent transparent transparent currentColor;
}
.vertical div.jx-arrow.jx-left {
left: 0px;
top: 2px;
border-style: solid;
border-width: 0px 8px 8px 8px;
border-color: transparent transparent currentColor transparent;
}
.vertical div.jx-arrow.jx-right {
right: 0px;
top: auto;
bottom: 2px;
border-style: solid;
border-width: 8px 8px 0 8px;
border-color: currentColor transparent transparent transparent;
}
div.jx-handle:hover div.jx-arrow.jx-left,
div.jx-handle:active div.jx-arrow.jx-left {
left: -1px;
}
div.jx-handle:hover div.jx-arrow.jx-right,
div.jx-handle:active div.jx-arrow.jx-right {
right: -1px;
}
.vertical div.jx-handle:hover div.jx-arrow.jx-left,
.vertical div.jx-handle:active div.jx-arrow.jx-left {
left: 0px;
top: 0px;
}
.vertical div.jx-handle:hover div.jx-arrow.jx-right,
.vertical div.jx-handle:active div.jx-arrow.jx-right {
right: 0px;
bottom: 0px;
}
div.jx-image {
position: absolute;
height: 100%;
display: inline-block;
top: 0;
overflow: hidden;
-webkit-backface-visibility: hidden;
}
.vertical div.jx-image {
width: 100%;
left: 0;
top: auto;
}
div.jx-image img {
height: 100%;
width: auto;
z-index: 5;
position: absolute;
margin-bottom: 0;
max-height: none;
max-width: none;
max-height: initial;
max-width: initial;
}
.vertical div.jx-image img {
height: auto;
width: 100%;
}
div.jx-image.jx-left {
left: 0;
background-position: left;
}
div.jx-image.jx-left img {
left: 0;
}
div.jx-image.jx-right {
right: 0;
background-position: right;
}
div.jx-image.jx-right img {
right: 0;
bottom: 0;
}
.veritcal div.jx-image.jx-left {
top: 0;
background-position: top;
}
.veritcal div.jx-image.jx-left img {
top: 0;
}
.vertical div.jx-image.jx-right {
bottom: 0;
background-position: bottom;
}
.veritcal div.jx-image.jx-right img {
bottom: 0;
}
div.jx-image div.jx-label {
font-size: 1em;
padding: .25em .75em;
position: relative;
display: inline-block;
top: 0;
background-color: #000; /* IE 8 */
background-color: rgba(0,0,0,.7);
color: white;
z-index: 10;
white-space: nowrap;
line-height: 18px;
vertical-align: middle;
}
div.jx-image.jx-left div.jx-label {
float: left;
left: 0;
}
div.jx-image.jx-right div.jx-label {
float: right;
right: 0;
}
.vertical div.jx-image div.jx-label {
display: table;
position: absolute;
}
.vertical div.jx-image.jx-right div.jx-label {
left: 0;
bottom: 0;
top: auto;
}
div.jx-credit {
line-height: 1.1;
font-size: 0.75em;
}
div.jx-credit em {
font-weight: bold;
font-style: normal;
}
/* Animation */
div.jx-image.transition {
transition: width .5s ease;
}
div.jx-handle.transition {
transition: left .5s ease;
}
.vertical div.jx-image.transition {
transition: height .5s ease;
}
.vertical div.jx-handle.transition {
transition: top .5s ease;
}
/* Knight Lab Credit */
a.jx-knightlab {
background-color: #000; /* IE 8 */
background-color: rgba(0,0,0,.25);
bottom: 0;
display: table;
height: 14px;
line-height: 14px;
padding: 1px 4px 1px 5px;
position: absolute;
right: 0;
text-decoration: none;
z-index: 10;
}
a.jx-knightlab div.knightlab-logo {
display: inline-block;
vertical-align: middle;
height: 8px;
width: 8px;
background-color: #c34528;
transform: rotate(45deg);
-ms-transform: rotate(45deg);
-webkit-transform: rotate(45deg);
top: -1.25px;
position: relative;
cursor: pointer;
}
a.jx-knightlab:hover {
background-color: #000; /* IE 8 */
background-color: rgba(0,0,0,.35);
}
a.jx-knightlab:hover div.knightlab-logo {
background-color: #ce4d28;
}
a.jx-knightlab span.juxtapose-name {
display: table-cell;
margin: 0;
padding: 0;
font-family: Helvetica, Arial, sans-serif;
font-weight: 300;
color: white;
font-size: 10px;
padding-left: 0.375em;
vertical-align: middle;
line-height: normal;
text-shadow: none;
}
/* keyboard accessibility */
div.jx-controller:focus,
div.jx-image.jx-left div.jx-label:focus,
div.jx-image.jx-right div.jx-label:focus,
a.jx-knightlab:focus {
background: #eae34a;
color: #000;
}
a.jx-knightlab:focus span.juxtapose-name{
color: #000;
border: none;
}
+8
View File
File diff suppressed because one or more lines are too long