Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 shadowcz007
|
6cb9df700b | ||
|
shadowcz007
|
4fdda537b9 | ||
|
shadowcz007
|
cb6f32465a | ||
|
shadowcz007
|
c235e36cb4 | ||
|
shadowcz007
|
aaca440a94 | ||
|
shadow
|
1f57950a29 | ||
|
shadow
|
3d8855ec72 | ||
|
shadowcz007
|
ac9231d9f3 | ||
|
shadowcz007
|
37c0a56d89 | ||
|
shadowcz007
|
010915dac4 | ||
|
shadowcz007
|
83a8b3b970 | ||
|
shadowcz007
|
f130202aa1 | ||
|
shadowcz007
|
c8f6800bcd | ||
|
shadowcz007
|
078f9f5dd4 | ||
|
shadowcz007
|
c0de178c7d | ||
|
shadowcz007
|
1c767b538d | ||
|
shadowcz007
|
51aab44b5d | ||
|
shadowcz007
|
b8a0d4a67b |
@@ -1,8 +1,26 @@
|
||||
> 适配了最新版comfyui的py3.11 ，torch 2.1.2+cu121
|
||||
|
||||
|
||||
> 适配了最新版 comfyui 的 py3.11 ，torch 2.1.2+cu121
|
||||
> [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)
|
||||
|
||||
|
||||
##### `最新`：
|
||||
|
||||
ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。模型下载后，放置到 `models/llamafile/`
|
||||
|
||||
- 右键菜单支持 text-to-text，方便对 prompt 词补全
|
||||
|
||||
强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)，备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
|
||||
|
||||
- 右键菜单支持 image-to-text，使用多模态模型，多模态使用 [llava-phi-3-mini-gguf](https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main)，注意需要把llava-phi-3-mini-mmproj-f16.gguf也下载
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### `相关插件推荐`
|
||||
[comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab)
|
||||
|
||||
<!-- [comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab) -->
|
||||
|
||||
[comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)
|
||||
|
||||
@@ -12,26 +30,12 @@
|
||||
|
||||
<!-- [comfyui-CLIPSeg](https://github.com/shadowcz007/comfyui-CLIPSeg) -->
|
||||
|
||||
##### `最新`：
|
||||
ChatGPT节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
|
||||
## 🚀🚗🚚🏃 Workflow-to-APP
|
||||
|
||||
Model download,move to :```models/llamafile/```
|
||||
|
||||
强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)
|
||||
|
||||
备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
|
||||
|
||||
|
||||
> 右键菜单支持 text-to-text，方便对prompt词补全
|
||||
|
||||
|
||||
|
||||
|
||||
## 🚀🚗🚚🏃 Workflow-to-APP
|
||||
- 新增AppInfo节点，可以通过简单的配置，把workflow转变为一个Web APP。
|
||||
- 支持多个web app 切换
|
||||
- 发布为app的workflow，可以在右键里再次编辑了
|
||||
- web app可以设置分类，在comfyui右键菜单可以编辑更新web app
|
||||
- 新增 AppInfo 节点，可以通过简单的配置，把 workflow 转变为一个 Web APP。
|
||||
- 支持多个 web app 切换
|
||||
- 发布为 app 的 workflow，可以在右键里再次编辑了
|
||||
- web app 可以设置分类，在 comfyui 右键菜单可以编辑更新 web app
|
||||
- 支持动态提示
|
||||
|
||||
|
||||
@@ -41,7 +45,6 @@ Model download,move to :```models/llamafile/```
|
||||
- The workflow, which is now released as an app, can also be edited again by right-clicking.
|
||||
- The web app can be configured with categories, and the web app can be edited and updated in the right-click menu of ComfyUI.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -49,66 +52,67 @@ Model download,move to :```models/llamafile/```
|
||||
|
||||
|
||||
Example:
|
||||
|
||||
- workflow
|
||||
|
||||
[text-to-image](./workflow/Text-to-Image-app.json)
|
||||
|
||||
[text-to-image](./workflow/Text-to-Image-app.json)
|
||||
|
||||
APP-JSON:
|
||||
|
||||
- [text-to-image](./example/Text-to-Image_3.json)
|
||||
- [image-to-image](./example/Image-to-Image_2.json)
|
||||
- text-to-text
|
||||
|
||||
> 暂时支持 9 种节点作为界面上的输入节点：Load Image、VHS_LoadVideo、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
|
||||
> 暂时支持 9 种节点作为界面上的输入节点：Load Image、VHS*LoadVideo、CLIPTextEncode、PromptSlide、TextInput*、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
|
||||
|
||||
> 输出节点：PreviewImage 、SaveImage、ShowTextForGPT、VHS_VideoCombine、PromptImage
|
||||
|
||||
> seed统一输入控件，支持：SamplerCustom、KSampler
|
||||
> seed 统一输入控件，支持：SamplerCustom、KSampler
|
||||
|
||||
> 配套[ps插件](https://github.com/shadowcz007/comfyui-ps-plugin)
|
||||
> 配套[ps 插件](https://github.com/shadowcz007/comfyui-ps-plugin)
|
||||
|
||||
> 如果遇到上传图片不成功，请检查下：局域网或者是云服务，请使用https，端口8189这个服务（ 感谢 @Damien 反馈问题）
|
||||
> 如果遇到上传图片不成功，请检查下：局域网或者是云服务，请使用 https，端口 8189 这个服务（ 感谢 @Damien 反馈问题）
|
||||
|
||||
> If you encounter difficulties in uploading images, please check the following: for local network or cloud services, please use HTTPS and the service on port 8189. (Thanks to @Damien for reporting the issue.)
|
||||
|
||||
## 🏃🚗🚚🚀 Real-time Design
|
||||
|
||||
|
||||
## 🏃🚗🚚🚀 Real-time Design
|
||||
> ScreenShareNode & FloatingVideoNode. Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
|
||||
|
||||
|
||||
|
||||
https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43e-410a-ab3a-1952b7b4e7da
|
||||
|
||||
|
||||
<!-- [ScreenShareNode](./workflow/2-screeshare.json) -->
|
||||
|
||||
[ScreenShareNode & FloatingVideoNode](./workflow/3-FloatVideo-workflow.json)
|
||||
|
||||
!! Please use the address with HTTPS (https://127.0.0.1).
|
||||
|
||||
|
||||
### SpeechRecognition & SpeechSynthesis
|
||||
|
||||
|
||||
|
||||
[Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
|
||||
|
||||
### GPT
|
||||
> Support for calling multiple GPTs.Local LLM（llama.cpp）、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
|
||||
|
||||
> Support for calling multiple GPTs.Local LLM（llama.cpp）、 ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
|
||||
|
||||
|
||||
|
||||
[workflow-5](./workflow/5-gpt-workflow.json)
|
||||
|
||||
最新：ChatGPT 节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
|
||||
|
||||
最新：ChatGPT节点支持 Local LLM（llama.cpp），Phi3、llama3 都可以直接一个节点运行了。
|
||||
|
||||
Model download,move to :```models/llamafile/```
|
||||
Model download,move to :`models/llamafile/`
|
||||
|
||||
强烈推荐：[Phi-3-mini-4k-instruct-GGUF](https://huggingface.co/lmstudio-community/Phi-3-mini-4k-instruct-GGUF/tree/main)
|
||||
|
||||
备选：[llama3_if_ai_sdpromptmkr_q2k](https://hf-mirror.com/impactframes/llama3_if_ai_sdpromptmkr_q2k/tree/main)
|
||||
|
||||
|
||||
> 如果碰到安装失败，可以尝试手动安装
|
||||
|
||||
```
|
||||
../../../python_embeded/python.exe -s -m pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
|
||||
|
||||
@@ -116,20 +120,23 @@ Model download,move to :```models/llamafile/```
|
||||
|
||||
```
|
||||
|
||||
> [Mac](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)
|
||||
> [Mac](https://llama-cpp-python.readthedocs.io/en/latest/install/macos/)
|
||||
|
||||
```
|
||||
pip uninstall llama-cpp-python -y
|
||||
CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --no-cache-dir
|
||||
pip install 'llama-cpp-python[server]'
|
||||
```
|
||||
|
||||
```
|
||||
pip install llama-cpp-python \
|
||||
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
> PromptSlide
|
||||
|
||||
> 
|
||||
|
||||
<!--  -->
|
||||
|
||||
@@ -143,20 +150,20 @@ pip install llama-cpp-python \
|
||||
|
||||
> PromptImage & PromptSimplification,Assist in simplifying prompt words, comparing images and prompt word nodes.
|
||||
|
||||
> ChinesePrompt && PromptGenerate，中文prompt节点，直接用中文书写你的prompt
|
||||
> ChinesePrompt && PromptGenerate，中文 prompt 节点，直接用中文书写你的 prompt
|
||||
|
||||
|
||||
|
||||
|
||||
### Layers
|
||||
|
||||
> A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### 3D
|
||||
|
||||
|
||||
|
||||
[workflow](./assets/Image-to-3D_1.json)
|
||||
@@ -164,44 +171,44 @@ pip install llama-cpp-python \
|
||||
|
||||
[workflow](./workflow/3D-workflow.json)
|
||||
|
||||
|
||||
### Image
|
||||
|
||||
#### LoadImagesToBatch
|
||||
> Upload multiple images for batch input into the IP adapter.
|
||||
|
||||
> Upload multiple images for batch input into the IP adapter.
|
||||
|
||||
#### LoadImagesFromLocal
|
||||
> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.
|
||||
|
||||
> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.
|
||||
|
||||
|
||||
|
||||
[workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)
|
||||
|
||||
#### LoadImagesFromURL
|
||||
|
||||
> Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.
|
||||
|
||||
|
||||
### Style
|
||||
> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
|
||||
|
||||
> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
|
||||
|
||||
|
||||
|
||||
> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
|
||||
|
||||
> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
|
||||
|
||||
### Utils
|
||||
|
||||
> The Color node provides a color picker for easy color selection, the Font node offers built-in font selection for use with TextImage to generate text images, and the DynamicDelayByText node allows delayed execution based on the length of the input text.
|
||||
|
||||
- [添加了DynamicDelayByText功能，可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
|
||||
- [添加了 DynamicDelayByText 功能，可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
|
||||
|
||||
- [Added DynamicDelayByText, enabling delayed execution based on input text length.](./workflow/audio-chatgpt-workflow.json)
|
||||
|
||||
- [使用CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)
|
||||
- [使用 CkptNames 对比不同的模型效果](./workflow/ckpts-image-workflow.json)
|
||||
|
||||
- [CkptNames compare the effects of different models.](./workflow/ckpts-image-workflow.json)
|
||||
|
||||
|
||||
|
||||
### Other Nodes
|
||||
|
||||
|
||||
@@ -209,33 +216,27 @@ pip install llama-cpp-python \
|
||||
|
||||
[workflow-1](./workflow/1-workflow.json)
|
||||
|
||||
|
||||
|
||||
> TransparentImage
|
||||
|
||||
|
||||
|
||||
|
||||
> FeatheredMask、SmoothMask
|
||||
|
||||
Add edges to an image.
|
||||
|
||||
|
||||
|
||||
|
||||
> LaMaInpainting
|
||||
|
||||
from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainting)
|
||||
|
||||
|
||||
> rembgNode
|
||||
|
||||
"briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"
|
||||
|
||||
*** briarmbg *** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
|
||||
**_ briarmbg _** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
|
||||
|
||||
|
||||
### Improvement
|
||||
### Improvement
|
||||
|
||||
- Add "help" option to the context menu for each node.
|
||||
- Add "Nodes Map" option to the global context menu.
|
||||
@@ -246,23 +247,21 @@ An improvement has been made to directly redirect to GitHub to search for missin
|
||||
|
||||
|
||||
|
||||
|
||||
### Models
|
||||
|
||||
* [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in ```models/triposr```
|
||||
- [Download TripoSR](https://huggingface.co/stabilityai/TripoSR/blob/main/model.ckpt) and place it in `models/triposr`
|
||||
|
||||
* [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main) and place it in ```models/triposr/facebook/dino-vitb16```
|
||||
- [Download facebook/dino-vitb16](https://huggingface.co/facebook/dino-vitb16/tree/main) and place it in `models/triposr/facebook/dino-vitb16`
|
||||
|
||||
[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:`models/rembg`
|
||||
|
||||
[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:```models/rembg```
|
||||
[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : `models/lama`
|
||||
|
||||
[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : ```models/lama```
|
||||
[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :`models/clip_interrogator/Salesforce/blip-image-captioning-base`
|
||||
|
||||
[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to :```models/clip_interrogator/Salesforce/blip-image-captioning-base```
|
||||
[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:`models/prompt_generator/text2image-prompt-generator`
|
||||
|
||||
[Download succinctly/text2image-prompt-generator](https://huggingface.co/succinctly/text2image-prompt-generator/tree/main),move to:```models/prompt_generator/text2image-prompt-generator```
|
||||
|
||||
[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:```models/prompt_generator/opus-mt-zh-en```
|
||||
[Download Helsinki-NLP/opus-mt-zh-en](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main),move to:`models/prompt_generator/opus-mt-zh-en`
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -278,40 +277,35 @@ git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
|
||||
Install the requirements:
|
||||
|
||||
run directly:
|
||||
|
||||
```
|
||||
cd ComfyUI/custom_nodes/comfyui-mixlab-nodes
|
||||
install.bat
|
||||
```
|
||||
|
||||
or install the requirements using:
|
||||
|
||||
```
|
||||
../../../python_embeded/python.exe -s -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
If you are using a venv, make sure you have it activated before installation and use:
|
||||
|
||||
```
|
||||
pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Chinese community
|
||||
访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab无界社区
|
||||
|
||||
访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab 无界社区
|
||||
|
||||
####
|
||||
|
||||
|
||||
####
|
||||
File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
|
||||
|
||||
|
||||
|
||||
|
||||
#### discussions:
|
||||
[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
|
||||
|
||||
[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
|
||||
|
||||
<picture>
|
||||
<source
|
||||
@@ -331,4 +325,3 @@ File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
|
||||
src="https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date"
|
||||
/>
|
||||
</picture>
|
||||
|
||||
|
||||
+44
-14
@@ -20,6 +20,7 @@ except:
|
||||
|
||||
llama_port=None
|
||||
llama_model=""
|
||||
llama_chat_format=""
|
||||
|
||||
try:
|
||||
from .nodes.ChatGPT import get_llama_models,get_llama_model_path,llama_cpp_client
|
||||
@@ -651,9 +652,9 @@ async def post_prompt_result(request):
|
||||
|
||||
|
||||
async def start_local_llm(data):
|
||||
global llama_port,llama_model
|
||||
if llama_port and llama_model:
|
||||
return {"port":llama_port,"model":llama_model}
|
||||
global llama_port,llama_model,llama_chat_format
|
||||
if llama_port and llama_model and llama_chat_format:
|
||||
return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
|
||||
|
||||
import threading
|
||||
import uvicorn
|
||||
@@ -671,7 +672,29 @@ async def start_local_llm(data):
|
||||
|
||||
elif "model" in data:
|
||||
model=get_llama_model_path(data['model'])
|
||||
|
||||
n_gpu_layers=-1
|
||||
|
||||
if "n_gpu_layers" in data:
|
||||
n_gpu_layers=data['n_gpu_layers']
|
||||
|
||||
|
||||
chat_format="chatml"
|
||||
|
||||
model_alias=os.path.basename(model)
|
||||
|
||||
# 多模态
|
||||
clip_model_path=None
|
||||
|
||||
prefix = "llava-phi-3-mini"
|
||||
file_name = prefix+"-mmproj-"
|
||||
if model_alias.startswith(prefix):
|
||||
for file in os.listdir(os.path.dirname(model)):
|
||||
if file.startswith(file_name):
|
||||
clip_model_path=os.path.join(os.path.dirname(model),file)
|
||||
chat_format='llava-1-5'
|
||||
print('#clip_model_path',chat_format,clip_model_path)
|
||||
|
||||
|
||||
address="127.0.0.1"
|
||||
port=9090
|
||||
@@ -693,9 +716,12 @@ async def start_local_llm(data):
|
||||
model_settings=[
|
||||
ModelSettings(
|
||||
model=model,
|
||||
n_gpu_layers=-1,
|
||||
model_alias=os.path.basename(model),
|
||||
n_gpu_layers=n_gpu_layers,
|
||||
n_ctx=4098,
|
||||
chat_format="chatml"
|
||||
chat_format=chat_format,
|
||||
embedding=False,
|
||||
clip_model_path=clip_model_path
|
||||
)])
|
||||
|
||||
def run_uvicorn():
|
||||
@@ -715,8 +741,9 @@ async def start_local_llm(data):
|
||||
|
||||
llama_port=port
|
||||
llama_model=data['model']
|
||||
llama_chat_format=chat_format
|
||||
|
||||
return {"port":llama_port,"model":llama_model}
|
||||
return {"port":llama_port,"model":llama_model,"chat_format":llama_chat_format}
|
||||
|
||||
# llam服务的开启
|
||||
@routes.post('/mixlab/start_llama')
|
||||
@@ -733,19 +760,20 @@ async def my_hander_method(request):
|
||||
|
||||
return web.json_response(result)
|
||||
|
||||
# parser.add_argument("--llm-model", type=str, help="Path to Model file (gguf). Run the Local LLM by MixlabNodes")
|
||||
# if args.llm_model and os.path.exists(args.llm_model):
|
||||
# start_local_llm({
|
||||
# "model_path":args.llm_model
|
||||
# })
|
||||
# print("Local LLM Start")
|
||||
|
||||
# 重启服务
|
||||
@routes.post('/mixlab/re_start')
|
||||
def re_start(request):
|
||||
try:
|
||||
sys.stdout.close_log()
|
||||
except Exception as e:
|
||||
pass
|
||||
return os.execv(sys.executable, [sys.executable] + sys.argv)
|
||||
|
||||
|
||||
|
||||
# 导入节点
|
||||
from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
|
||||
from .nodes.ImageNode import LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
|
||||
from .nodes.ImageNode import ComparingTwoFrames,LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
|
||||
# from .nodes.Vae import VAELoader,VAEDecode
|
||||
from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
|
||||
|
||||
@@ -806,6 +834,7 @@ NODE_CLASS_MAPPINGS = {
|
||||
# "VAELoaderConsistencyDecoder":VAELoader,
|
||||
"SaveImageToLocal":SaveImageToLocal,
|
||||
"SaveImageAndMetadata_":SaveImageAndMetadata,
|
||||
"ComparingTwoFrames_":ComparingTwoFrames,
|
||||
# "VAEDecodeConsistencyDecoder":VAEDecode,
|
||||
"ScreenShare":ScreenShareNode,
|
||||
"FloatingVideo":FloatingVideo,
|
||||
@@ -860,6 +889,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"IntNumber":"Int Input ♾️MixlabApp",
|
||||
"ImagesPrompt_":"Images Input ♾️MixlabApp",
|
||||
"SaveImageAndMetadata_":"Save Image Output ♾️MixlabApp",
|
||||
"ComparingTwoFrames_":"Comparing Two Frames ♾️MixlabApp",
|
||||
"ResizeImageMixlab":"Resize Image ♾️Mixlab",
|
||||
"RandomPrompt": "Random Prompt ♾️Mixlab",
|
||||
"PromptImage":"Output Prompt and Image ♾️Mixlab",
|
||||
|
||||
@@ -2983,6 +2983,65 @@ class SaveImageAndMetadata:
|
||||
|
||||
return { "ui": { "images": results } }
|
||||
|
||||
class ComparingTwoFrames:
|
||||
def __init__(self):
|
||||
self.output_dir = folder_paths.get_output_directory()
|
||||
self.type = "output"
|
||||
self.prefix_append = "ComparingTwoFrames"
|
||||
self.compress_level = 4
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required":
|
||||
{"before_image": ("IMAGE", ),
|
||||
"after_image": ("IMAGE", )
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ()
|
||||
FUNCTION = "comparingImages"
|
||||
|
||||
OUTPUT_NODE = True
|
||||
|
||||
CATEGORY = "♾️Mixlab/Output"
|
||||
|
||||
def comparingImages(self, before_image,after_image):
|
||||
filename_prefix = self.prefix_append
|
||||
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(
|
||||
filename_prefix, self.output_dir, after_image[0].shape[1], after_image[0].shape[0])
|
||||
|
||||
bresults = list()
|
||||
|
||||
for bimage in before_image:
|
||||
i = 255. * bimage.cpu().numpy()
|
||||
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
|
||||
|
||||
file = f"{filename}_{counter:05}_.png"
|
||||
img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
|
||||
bresults.append({
|
||||
"filename": file,
|
||||
"subfolder": subfolder,
|
||||
"type": self.type
|
||||
})
|
||||
counter += 1
|
||||
|
||||
|
||||
results = list()
|
||||
for aimage in after_image:
|
||||
i = 255. * aimage.cpu().numpy()
|
||||
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
|
||||
|
||||
file = f"{filename}_{counter:05}_.png"
|
||||
img.save(os.path.join(full_output_folder, file), pnginfo=None, compress_level=self.compress_level)
|
||||
results.append({
|
||||
"filename": file,
|
||||
"subfolder": subfolder,
|
||||
"type": self.type
|
||||
})
|
||||
counter += 1
|
||||
|
||||
return { "ui": { "after_images": results,"before_images":bresults } }
|
||||
|
||||
class ImageColorTransfer:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
|
||||
@@ -36,7 +36,6 @@ async function* completion (url, messages, controller) {
|
||||
break
|
||||
}
|
||||
|
||||
|
||||
// Add any leftover data to the current chunk of data
|
||||
const text = leftover + decoder.decode(result.value)
|
||||
|
||||
@@ -64,14 +63,14 @@ async function* completion (url, messages, controller) {
|
||||
if (result.data) {
|
||||
result.data = JSON.parse(result.data)
|
||||
// console.log('#result.data',result.data)
|
||||
|
||||
content += result.data.choices[0].delta?.content||''
|
||||
|
||||
content += result.data.choices[0].delta?.content || ''
|
||||
|
||||
// yield
|
||||
yield result
|
||||
|
||||
// if we got a stop token from server, we will break here
|
||||
if (result.data.choices[0].finish_reason=="stop") {
|
||||
if (result.data.choices[0].finish_reason == 'stop') {
|
||||
if (result.data.generation_settings) {
|
||||
// generation_settings = result.data.generation_settings;
|
||||
}
|
||||
@@ -96,11 +95,10 @@ async function* completion (url, messages, controller) {
|
||||
export async function completion_ (url, messages, controller, callback) {
|
||||
let request = await completion(url, messages, controller)
|
||||
for await (const chunk of request) {
|
||||
|
||||
let content=chunk.data.choices[0].delta.content||""
|
||||
if(chunk.data.choices[0].role=="assistant"){
|
||||
let content = chunk.data.choices[0].delta.content || ''
|
||||
if (chunk.data.choices[0].role == 'assistant') {
|
||||
//开始
|
||||
content=""
|
||||
content = ''
|
||||
}
|
||||
|
||||
if (callback) callback(content)
|
||||
|
||||
@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
|
||||
const repoOwner = 'shadowcz007' // 替换为仓库的所有者
|
||||
const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
|
||||
|
||||
const version = 'v0.25.1'
|
||||
const version = 'v0.26.0'
|
||||
|
||||
fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
|
||||
.then(response => response.json())
|
||||
|
||||
@@ -705,7 +705,9 @@ app.registerExtension({
|
||||
|
||||
async beforeRegisterNodeDef (nodeType, nodeData, app) {
|
||||
if (nodeType.comfyClass == 'LoadImagesToBatch') {
|
||||
|
||||
const orig_nodeCreated = nodeType.prototype.onNodeCreated
|
||||
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
orig_nodeCreated?.apply(this, arguments)
|
||||
|
||||
@@ -841,3 +843,150 @@ app.registerExtension({
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
// 如何引入css
|
||||
app.registerExtension({
|
||||
name: 'Mixlab.output.ComparingTwoFrames_',
|
||||
init () {
|
||||
$el('link', {
|
||||
rel: 'stylesheet',
|
||||
href: '/extensions/comfyui-mixlab-nodes/lib/juxtapose.css',
|
||||
parent: document.head
|
||||
})
|
||||
|
||||
$el('style', {
|
||||
textContent: `
|
||||
.juxtapose-name{
|
||||
display: none!important;
|
||||
}
|
||||
`,
|
||||
parent: document.body
|
||||
})
|
||||
},
|
||||
async beforeRegisterNodeDef (nodeType, nodeData, app) {
|
||||
if (nodeType.comfyClass == 'ComparingTwoFrames_') {
|
||||
const onNodeCreated = nodeType.prototype.onNodeCreated
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const r = onNodeCreated
|
||||
? onNodeCreated.apply(this, arguments)
|
||||
: undefined
|
||||
|
||||
this.size = [400, this.size[1]]
|
||||
console.log('##onNodeCreated', this)
|
||||
const widget = {
|
||||
type: 'div',
|
||||
name: 'preview',
|
||||
draw (ctx, node, widget_width, y, widget_height) {
|
||||
Object.assign(
|
||||
this.div.style,
|
||||
get_position_style(ctx, 400, 44, node.size[1])
|
||||
)
|
||||
},
|
||||
serialize: false
|
||||
}
|
||||
|
||||
widget.div = $el('div', {})
|
||||
|
||||
document.body.appendChild(widget.div)
|
||||
this.addCustomWidget(widget)
|
||||
this.serialize_widgets = true //需要保存参数
|
||||
|
||||
|
||||
const onRemoved = this.onRemoved
|
||||
this.onRemoved = () => {
|
||||
widget.div.remove()
|
||||
return onRemoved?.()
|
||||
}
|
||||
|
||||
|
||||
return r
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments)
|
||||
console.log('##onExecuted', this, message)
|
||||
|
||||
this.widgets[0].div.id = 'mix_comparingtowframes_' + this.id
|
||||
|
||||
let after_image = message.after_images[0]
|
||||
let before_image = message.before_images[0]
|
||||
|
||||
after_image = `${window.location.protocol}//${
|
||||
window.location.hostname
|
||||
}:${window.location.port}/view?filename=${encodeURIComponent(
|
||||
after_image.filename
|
||||
)}&type=${after_image.type}&subfolder=${encodeURIComponent(
|
||||
after_image.subfolder
|
||||
)}&t=${+new Date()}`
|
||||
|
||||
before_image = `${window.location.protocol}//${
|
||||
window.location.hostname
|
||||
}:${window.location.port}/view?filename=${encodeURIComponent(
|
||||
before_image.filename
|
||||
)}&type=${before_image.type}&subfolder=${encodeURIComponent(
|
||||
before_image.subfolder
|
||||
)}&t=${+new Date()}`
|
||||
|
||||
this.widgets[0].div.innerHTML = ''
|
||||
|
||||
let slider = new juxtapose.JXSlider(
|
||||
'#mix_comparingtowframes_' + this.id,
|
||||
[
|
||||
{
|
||||
src: before_image,
|
||||
label: 'Before'
|
||||
},
|
||||
{
|
||||
src: after_image,
|
||||
label: 'After'
|
||||
}
|
||||
],
|
||||
{
|
||||
animate: true,
|
||||
showLabels: true,
|
||||
showCredits: false,
|
||||
startingPosition: '50%',
|
||||
makeResponsive: false
|
||||
}
|
||||
)
|
||||
|
||||
this.widgets_values = [
|
||||
{
|
||||
src: before_image,
|
||||
label: 'Before'
|
||||
},
|
||||
{
|
||||
src: after_image,
|
||||
label: 'After'
|
||||
}
|
||||
]
|
||||
this.size=[this.size[0],300]
|
||||
}
|
||||
}
|
||||
},
|
||||
async loadedGraphNode (node, app) {
|
||||
// console.log('##loadedGraphNode', node)
|
||||
if (node.type === 'ComparingTwoFrames_') {
|
||||
// node.widgets[0].div.id = 'mix_comparingtowframes_' + node.id
|
||||
// if (node.widgets_values && node.widgets_values[0]) {
|
||||
// node.widgets[0].div.innerHTML = ''
|
||||
|
||||
// let slider = new juxtapose.JXSlider(
|
||||
// '#mix_comparingtowframes_' + node.id,
|
||||
// node.widgets_values,
|
||||
// {
|
||||
// animate: true,
|
||||
// showLabels: true,
|
||||
// showCredits: false,
|
||||
// startingPosition: '50%',
|
||||
// makeResponsive: false
|
||||
// }
|
||||
// )
|
||||
// }
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
+320
-21
@@ -11,6 +11,20 @@ import { smart_init, addSmartMenu } from './smart_connect.js'
|
||||
|
||||
import { completion_ } from './chat.js'
|
||||
|
||||
function showTextByLanguage (key, json) {
|
||||
// 获取浏览器语言
|
||||
var language = navigator.language
|
||||
// 判断是否为中文
|
||||
if (
|
||||
language.indexOf('zh') !== -1 ||
|
||||
(language.indexOf('cn') !== -1 && json[key])
|
||||
) {
|
||||
return json[key]
|
||||
} else {
|
||||
return key
|
||||
}
|
||||
}
|
||||
|
||||
//系统prompt
|
||||
const systemPrompt = `You are a prompt creator, your task is to create prompts for the user input request, the prompts are image descriptions that include keywords for (an adjective, type of image, framing/composition, subject, subject appearance/action, environment, lighting situation, details of the shoot/illustration, visuals aesthetics and artists), brake keywords by comas, provide high quality, non-verboose, coherent, brief, concise, and not superfluous prompts, the subject from the input request must be included verbatim on the prompt,the prompt is english`
|
||||
|
||||
@@ -40,6 +54,11 @@ async function get_llamafile_models () {
|
||||
}
|
||||
// 运行llama
|
||||
async function start_llama (model = 'Phi-3-mini-4k-instruct-Q5_K_S.gguf') {
|
||||
let n_gpu_layers = -1
|
||||
try {
|
||||
n_gpu_layers = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
|
||||
} catch (error) {}
|
||||
|
||||
try {
|
||||
const response = await fetch('/mixlab/start_llama', {
|
||||
method: 'POST',
|
||||
@@ -47,22 +66,57 @@ async function start_llama (model = 'Phi-3-mini-4k-instruct-Q5_K_S.gguf') {
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model
|
||||
model,
|
||||
n_gpu_layers
|
||||
})
|
||||
})
|
||||
|
||||
const data = await response.json();
|
||||
if(data.llama_cpp_error){
|
||||
return
|
||||
const data = await response.json()
|
||||
if (data.llama_cpp_error) {
|
||||
return
|
||||
}
|
||||
|
||||
return {
|
||||
url: `http://${window.location.hostname}:${data.port}`,
|
||||
model: data.model,
|
||||
chat_format: data.chat_format
|
||||
}
|
||||
|
||||
return { url: `http://127.0.0.1:${data.port}`, model: data.model }
|
||||
|
||||
} catch (error) {
|
||||
console.error(error)
|
||||
}
|
||||
}
|
||||
|
||||
function resizeImage (base64Image) {
|
||||
var img = new Image()
|
||||
var canvas = document.createElement('canvas')
|
||||
var ctx = canvas.getContext('2d')
|
||||
return new Promise((res, rej) => {
|
||||
img.onload = function () {
|
||||
// 等比例缩放图片
|
||||
var width = img.width
|
||||
var height = img.height
|
||||
var max_width = 768
|
||||
if (width > max_width) {
|
||||
height *= max_width / width
|
||||
width = max_width
|
||||
}
|
||||
|
||||
// 设置canvas尺寸
|
||||
canvas.width = width
|
||||
canvas.height = height
|
||||
|
||||
// 在canvas上绘制图片
|
||||
ctx.drawImage(img, 0, 0, width, height)
|
||||
|
||||
// 将canvas转换为base64图片数据
|
||||
var canvasData = canvas.toDataURL()
|
||||
res(canvasData) // canvas转换后的base64图片数据
|
||||
}
|
||||
|
||||
img.src = base64Image
|
||||
})
|
||||
}
|
||||
|
||||
// 菜单入口
|
||||
async function createMenu () {
|
||||
const menu = document.querySelector('.comfy-menu')
|
||||
@@ -610,13 +664,28 @@ app.showMissingNodesError = async function (
|
||||
// console.log('#nodesMap', nodesMap)
|
||||
// console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
|
||||
this.ui.dialog.show(
|
||||
`When loading the graph, the following node types were not found: <ul>${missingNodeGithub(
|
||||
missingNodeTypes,
|
||||
nodesMap
|
||||
).join('')}</ul>${
|
||||
hasAddedNodes
|
||||
? 'Nodes that have failed to load will show as red on the graph.'
|
||||
: ''
|
||||
`<a style="color: white;
|
||||
font-size: 18px;
|
||||
font-weight: 800;
|
||||
letter-spacing: 2px;
|
||||
font-family: sans-serif;
|
||||
}"
|
||||
href="https://discord.gg/cXs9vZSqeK" target="_blank">${showTextByLanguage(
|
||||
'Welcome to Mixlab nodes discord, seeking help.',
|
||||
{
|
||||
'Welcome to Mixlab nodes discord, seeking help.':
|
||||
'寻求帮助，加入Mixlab nodes交流频道'
|
||||
}
|
||||
)}</a><br><br>${showTextByLanguage(
|
||||
'When loading the graph, the following node types were not found:',
|
||||
{
|
||||
'When loading the graph, the following node types were not found:':
|
||||
'缺少以下节点：'
|
||||
}
|
||||
)}
|
||||
|
||||
<ul>${missingNodeGithub(missingNodeTypes, nodesMap).join('')}</ul>${
|
||||
hasAddedNodes ? '' : ''
|
||||
}`
|
||||
)
|
||||
this.logging.addEntry('Comfy.App', 'warn', {
|
||||
@@ -768,15 +837,73 @@ function createModelsModal (models) {
|
||||
user-select: none;
|
||||
`
|
||||
|
||||
headTitleElement.textContent = 'Models'
|
||||
// headTitleElement.href = 'https://github.com/shadowcz007/comfyui-mixlab-nodes'
|
||||
// headTitleElement.target = '_blank'
|
||||
const linkIcon = document.createElement('small')
|
||||
linkIcon.textContent = '自动开启'
|
||||
linkIcon.textContent = showTextByLanguage('Auto Open', {
|
||||
'Auto Open': '自动开启'
|
||||
})
|
||||
linkIcon.style.padding = '4px'
|
||||
|
||||
headTitleElement.appendChild(linkIcon)
|
||||
headerElement.appendChild(headTitleElement)
|
||||
const n_gpu = document.createElement('input')
|
||||
n_gpu.type = 'number'
|
||||
n_gpu.setAttribute('min', -1)
|
||||
n_gpu.setAttribute('max', 9999)
|
||||
|
||||
n_gpu.style = `color: var(--input-text);
|
||||
background-color: var(--comfy-input-bg);
|
||||
border-radius: 8px;
|
||||
border-color: var(--border-color);
|
||||
height: 26px;
|
||||
padding: 4px 10px;
|
||||
width: 48px;
|
||||
margin-left: 12px;`
|
||||
if (localStorage.getItem('_mixlab_llama_n_gpu')) {
|
||||
n_gpu.value = parseInt(localStorage.getItem('_mixlab_llama_n_gpu'))
|
||||
} else {
|
||||
n_gpu.value = -1
|
||||
localStorage.setItem('_mixlab_llama_n_gpu', -1)
|
||||
}
|
||||
|
||||
const n_gpu_p = document.createElement('p')
|
||||
n_gpu_p.innerText = 'n_gpu_layers'
|
||||
|
||||
const n_gpu_div = document.createElement('div')
|
||||
n_gpu_div.style = `display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 12px;`
|
||||
n_gpu_div.appendChild(n_gpu_p)
|
||||
n_gpu_div.appendChild(n_gpu)
|
||||
|
||||
const title = document.createElement('p')
|
||||
title.innerText = 'Models'
|
||||
title.style = `font-size: 18px;
|
||||
margin-right: 8px;`
|
||||
|
||||
const left_d = document.createElement('div')
|
||||
left_d.style = `display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 12px;`
|
||||
left_d.appendChild(title)
|
||||
|
||||
left_d.appendChild(linkIcon)
|
||||
|
||||
headTitleElement.appendChild(left_d)
|
||||
|
||||
headTitleElement.appendChild(n_gpu_div)
|
||||
|
||||
//重启
|
||||
const reStart = document.createElement('small')
|
||||
reStart.textContent = showTextByLanguage('restart', {
|
||||
restart: '重启'
|
||||
})
|
||||
|
||||
reStart.style.padding = '4px'
|
||||
|
||||
headTitleElement.appendChild(reStart)
|
||||
|
||||
if (localStorage.getItem('_mixlab_auto_llama_open')) {
|
||||
linkIcon.style.backgroundColor = '#66ff6c'
|
||||
linkIcon.style.color = 'black'
|
||||
@@ -794,6 +921,19 @@ function createModelsModal (models) {
|
||||
}
|
||||
})
|
||||
|
||||
reStart.addEventListener('click', e => {
|
||||
e.stopPropagation()
|
||||
div.remove()
|
||||
fetch('mixlab/re_start', {
|
||||
method: 'POST'
|
||||
})
|
||||
})
|
||||
|
||||
n_gpu.addEventListener('click', e => {
|
||||
e.stopPropagation()
|
||||
localStorage.setItem('_mixlab_llama_n_gpu', n_gpu.value)
|
||||
})
|
||||
|
||||
modal.appendChild(headTitleElement)
|
||||
|
||||
// Create modal content area
|
||||
@@ -845,6 +985,20 @@ function createModelsModal (models) {
|
||||
}
|
||||
modal.appendChild(modalContent)
|
||||
|
||||
const helpInfo = document.createElement('a')
|
||||
helpInfo.textContent = showTextByLanguage('Help', {
|
||||
Help: '寻求帮助'
|
||||
})
|
||||
helpInfo.style = `text-align: center;
|
||||
display: block;
|
||||
padding: 8px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
color: white;`
|
||||
helpInfo.href="https://discord.gg/cXs9vZSqeK"
|
||||
helpInfo.target="_blank"
|
||||
modal.appendChild(helpInfo)
|
||||
|
||||
document.body.appendChild(div)
|
||||
}
|
||||
|
||||
@@ -1158,6 +1312,32 @@ function drawBadge (node, orig, restArgs) {
|
||||
return r
|
||||
}
|
||||
|
||||
function convertImageUrlToBase64 (imageUrl) {
|
||||
return fetch(imageUrl)
|
||||
.then(response => response.blob())
|
||||
.then(blob => {
|
||||
return new Promise((resolve, reject) => {
|
||||
const reader = new FileReader()
|
||||
reader.onloadend = () => resolve(reader.result)
|
||||
reader.onerror = reject
|
||||
reader.readAsDataURL(blob)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
async function getSelectImageNode () {
|
||||
var nodes = app.canvas.selected_nodes
|
||||
let imageNode = null
|
||||
if (Object.keys(app.canvas.selected_nodes).length == 0) return
|
||||
for (var id in nodes) {
|
||||
if (nodes[id].imgs) {
|
||||
let base64 = await convertImageUrlToBase64(nodes[id].imgs[0].currentSrc)
|
||||
imageNode = await resizeImage(base64)
|
||||
}
|
||||
}
|
||||
return imageNode
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: 'Comfy.Mixlab.ui',
|
||||
init () {
|
||||
@@ -1202,8 +1382,10 @@ app.registerExtension({
|
||||
w => w.name === 'text' && typeof w.value == 'string'
|
||||
)[0]
|
||||
if (widget) {
|
||||
app.canvas.centerOnNode(node)
|
||||
|
||||
let controller = new AbortController()
|
||||
let ends = []
|
||||
let ends = [] //TODO 判断终止 <|im_start|>
|
||||
let userInput = widget.value
|
||||
widget.value = widget.value.trim()
|
||||
widget.value += '\n'
|
||||
@@ -1256,6 +1438,109 @@ app.registerExtension({
|
||||
}
|
||||
}
|
||||
|
||||
LGraphCanvas.prototype.image2text = async function (node) {
|
||||
let imageBase64 = await getSelectImageNode()
|
||||
|
||||
if (imageBase64) {
|
||||
// console.log('image2text')
|
||||
// 添加note 节点
|
||||
const NoteNode = LiteGraph.createNode('Note')
|
||||
NoteNode.title = `Image-to-Text ${node.id}`
|
||||
NoteNode.size = [NoteNode.size[0] + 100, NoteNode.size[1]]
|
||||
let widget = NoteNode.widgets[0]
|
||||
widget.value = ''
|
||||
|
||||
NoteNode.pos = [node.pos[0] + node.size[0] + 24, node.pos[1] - 48]
|
||||
|
||||
app.canvas.graph.add(NoteNode, false)
|
||||
app.canvas.centerOnNode(NoteNode)
|
||||
|
||||
let controller = new AbortController()
|
||||
let ends = []
|
||||
let userInput = widget.value
|
||||
widget.value = widget.value.trim()
|
||||
widget.value += '\n'
|
||||
|
||||
try {
|
||||
await completion_(
|
||||
window._mixlab_llamacpp.url + '/v1/chat/completions',
|
||||
[
|
||||
{
|
||||
role: 'system',
|
||||
content: localStorage.getItem('_mixlab_system_prompt')
|
||||
},
|
||||
// { role: 'user', content: userInput }
|
||||
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: imageBase64
|
||||
}
|
||||
},
|
||||
{ type: 'text', text: 'What’s in this image?' }
|
||||
]
|
||||
}
|
||||
],
|
||||
controller,
|
||||
t => {
|
||||
// console.log(t)
|
||||
widget.value += t
|
||||
|
||||
NoteNode.size[1] = widget.element.scrollHeight + 20
|
||||
widget.computedHeight = NoteNode.size[1]
|
||||
app.canvas.centerOnNode(NoteNode)
|
||||
}
|
||||
)
|
||||
} catch (error) {
|
||||
//是否要自动加载模型
|
||||
if (localStorage.getItem('_mixlab_auto_llama_open')) {
|
||||
let model = localStorage.getItem('_mixlab_llama_select')
|
||||
start_llama(model).then(async res => {
|
||||
window._mixlab_llamacpp = res
|
||||
document.body
|
||||
.querySelector('#mixlab_chatbot_by_llamacpp')
|
||||
.setAttribute('title', res.url)
|
||||
|
||||
await completion_(
|
||||
window._mixlab_llamacpp.url + '/v1/chat/completions',
|
||||
[
|
||||
{
|
||||
role: 'system',
|
||||
content: localStorage.getItem('_mixlab_system_prompt')
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'image_url',
|
||||
image_url: {
|
||||
url: imageBase64
|
||||
}
|
||||
},
|
||||
{ type: 'text', text: 'What’s in this image?' }
|
||||
]
|
||||
}
|
||||
],
|
||||
controller,
|
||||
t => {
|
||||
// console.log(t)
|
||||
widget.value += t
|
||||
NoteNode.size[1] = widget.element.scrollHeight + 20
|
||||
widget.computedHeight = NoteNode.size[1]
|
||||
app.canvas.centerOnNode(NoteNode)
|
||||
}
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
widget.value = widget.value.trim()
|
||||
}
|
||||
}
|
||||
|
||||
const getGroupMenuOptions = LGraphCanvas.prototype.getGroupMenuOptions // store the existing method
|
||||
LGraphCanvas.prototype.getGroupMenuOptions = function (node) {
|
||||
// replace it
|
||||
@@ -1402,7 +1687,7 @@ app.registerExtension({
|
||||
this.setDirty(true, true)
|
||||
}
|
||||
|
||||
const getNodeMenuOptions=LGraphCanvas.prototype.getNodeMenuOptions;
|
||||
const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions
|
||||
LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
|
||||
// replace it
|
||||
const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
|
||||
@@ -1433,7 +1718,7 @@ app.registerExtension({
|
||||
)
|
||||
|
||||
if (
|
||||
text_input&&
|
||||
text_input &&
|
||||
text_input.length == 0 &&
|
||||
text_widget &&
|
||||
text_widget.length == 1 &&
|
||||
@@ -1447,6 +1732,20 @@ app.registerExtension({
|
||||
} // and the callback
|
||||
})
|
||||
}
|
||||
|
||||
if (
|
||||
node.imgs &&
|
||||
node.imgs.length > 0 &&
|
||||
window._mixlab_llamacpp &&
|
||||
window._mixlab_llamacpp.chat_format === 'llava-1-5'
|
||||
) {
|
||||
opts.push({
|
||||
content: 'Image-to-Text ♾️Mixlab', // with a name
|
||||
callback: () => {
|
||||
LGraphCanvas.prototype.image2text(node)
|
||||
} // and the callback
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return [...opts, null, ...options] // and return the options
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
/* juxtapose - v1.2.2 - 2020-09-03
|
||||
* Copyright (c) 2020 Alex Duner and Northwestern University Knight Lab
|
||||
*/
|
||||
div.juxtapose {
|
||||
width: 100%;
|
||||
font-family: Helvetica, Arial, sans-serif;
|
||||
}
|
||||
|
||||
div.jx-slider {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
cursor: pointer;
|
||||
color: #f3f3f3;
|
||||
}
|
||||
|
||||
|
||||
div.jx-handle {
|
||||
position: absolute;
|
||||
height: 100%;
|
||||
width: 40px;
|
||||
cursor: col-resize;
|
||||
z-index: 15;
|
||||
margin-left: -20px;
|
||||
}
|
||||
|
||||
.vertical div.jx-handle {
|
||||
height: 40px;
|
||||
width: 100%;
|
||||
cursor: row-resize;
|
||||
margin-top: -20px;
|
||||
margin-left: 0;
|
||||
}
|
||||
|
||||
div.jx-control {
|
||||
height: 100%;
|
||||
margin-right: auto;
|
||||
margin-left: auto;
|
||||
width: 3px;
|
||||
background-color: currentColor;
|
||||
}
|
||||
|
||||
.vertical div.jx-control {
|
||||
height: 3px;
|
||||
width: 100%;
|
||||
background-color: currentColor;
|
||||
position: relative;
|
||||
top: 50%;
|
||||
transform: translateY(-50%);
|
||||
}
|
||||
|
||||
div.jx-controller {
|
||||
position: absolute;
|
||||
margin: auto;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
height: 60px;
|
||||
width: 9px;
|
||||
margin-left: -3px;
|
||||
background-color: currentColor;
|
||||
}
|
||||
|
||||
.vertical div.jx-controller {
|
||||
height: 9px;
|
||||
width: 100px;
|
||||
margin-left: auto;
|
||||
margin-right: auto;
|
||||
top: -3px;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
div.jx-arrow {
|
||||
position: absolute;
|
||||
margin: auto;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 0;
|
||||
height: 0;
|
||||
transition: all .2s ease;
|
||||
}
|
||||
|
||||
.vertical div.jx-arrow {
|
||||
position: absolute;
|
||||
margin: 0 auto;
|
||||
left: 0;
|
||||
right: 0;
|
||||
width: 0;
|
||||
height: 0;
|
||||
transition: all .2s ease;
|
||||
}
|
||||
|
||||
|
||||
div.jx-arrow.jx-left {
|
||||
left: 2px;
|
||||
border-style: solid;
|
||||
border-width: 8px 8px 8px 0;
|
||||
border-color: transparent currentColor transparent transparent;
|
||||
}
|
||||
|
||||
div.jx-arrow.jx-right {
|
||||
right: 2px;
|
||||
border-style: solid;
|
||||
border-width: 8px 0 8px 8px;
|
||||
border-color: transparent transparent transparent currentColor;
|
||||
}
|
||||
|
||||
.vertical div.jx-arrow.jx-left {
|
||||
left: 0px;
|
||||
top: 2px;
|
||||
border-style: solid;
|
||||
border-width: 0px 8px 8px 8px;
|
||||
border-color: transparent transparent currentColor transparent;
|
||||
}
|
||||
|
||||
.vertical div.jx-arrow.jx-right {
|
||||
right: 0px;
|
||||
top: auto;
|
||||
bottom: 2px;
|
||||
border-style: solid;
|
||||
border-width: 8px 8px 0 8px;
|
||||
border-color: currentColor transparent transparent transparent;
|
||||
}
|
||||
|
||||
div.jx-handle:hover div.jx-arrow.jx-left,
|
||||
div.jx-handle:active div.jx-arrow.jx-left {
|
||||
left: -1px;
|
||||
}
|
||||
|
||||
div.jx-handle:hover div.jx-arrow.jx-right,
|
||||
div.jx-handle:active div.jx-arrow.jx-right {
|
||||
right: -1px;
|
||||
}
|
||||
|
||||
.vertical div.jx-handle:hover div.jx-arrow.jx-left,
|
||||
.vertical div.jx-handle:active div.jx-arrow.jx-left {
|
||||
left: 0px;
|
||||
top: 0px;
|
||||
}
|
||||
|
||||
.vertical div.jx-handle:hover div.jx-arrow.jx-right,
|
||||
.vertical div.jx-handle:active div.jx-arrow.jx-right {
|
||||
right: 0px;
|
||||
bottom: 0px;
|
||||
}
|
||||
|
||||
|
||||
div.jx-image {
|
||||
position: absolute;
|
||||
height: 100%;
|
||||
display: inline-block;
|
||||
top: 0;
|
||||
overflow: hidden;
|
||||
-webkit-backface-visibility: hidden;
|
||||
}
|
||||
|
||||
.vertical div.jx-image {
|
||||
width: 100%;
|
||||
left: 0;
|
||||
top: auto;
|
||||
}
|
||||
|
||||
div.jx-image img {
|
||||
height: 100%;
|
||||
width: auto;
|
||||
z-index: 5;
|
||||
position: absolute;
|
||||
margin-bottom: 0;
|
||||
|
||||
max-height: none;
|
||||
max-width: none;
|
||||
max-height: initial;
|
||||
max-width: initial;
|
||||
}
|
||||
|
||||
.vertical div.jx-image img {
|
||||
height: auto;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
div.jx-image.jx-left {
|
||||
left: 0;
|
||||
background-position: left;
|
||||
}
|
||||
|
||||
div.jx-image.jx-left img {
|
||||
left: 0;
|
||||
}
|
||||
|
||||
div.jx-image.jx-right {
|
||||
right: 0;
|
||||
background-position: right;
|
||||
}
|
||||
|
||||
div.jx-image.jx-right img {
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
}
|
||||
|
||||
|
||||
.veritcal div.jx-image.jx-left {
|
||||
top: 0;
|
||||
background-position: top;
|
||||
}
|
||||
|
||||
.veritcal div.jx-image.jx-left img {
|
||||
top: 0;
|
||||
}
|
||||
|
||||
.vertical div.jx-image.jx-right {
|
||||
bottom: 0;
|
||||
background-position: bottom;
|
||||
}
|
||||
|
||||
.veritcal div.jx-image.jx-right img {
|
||||
bottom: 0;
|
||||
}
|
||||
|
||||
|
||||
div.jx-image div.jx-label {
|
||||
font-size: 1em;
|
||||
padding: .25em .75em;
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
top: 0;
|
||||
background-color: #000; /* IE 8 */
|
||||
background-color: rgba(0,0,0,.7);
|
||||
color: white;
|
||||
z-index: 10;
|
||||
white-space: nowrap;
|
||||
line-height: 18px;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
div.jx-image.jx-left div.jx-label {
|
||||
float: left;
|
||||
left: 0;
|
||||
}
|
||||
|
||||
div.jx-image.jx-right div.jx-label {
|
||||
float: right;
|
||||
right: 0;
|
||||
}
|
||||
|
||||
.vertical div.jx-image div.jx-label {
|
||||
display: table;
|
||||
position: absolute;
|
||||
}
|
||||
|
||||
.vertical div.jx-image.jx-right div.jx-label {
|
||||
left: 0;
|
||||
bottom: 0;
|
||||
top: auto;
|
||||
}
|
||||
|
||||
div.jx-credit {
|
||||
line-height: 1.1;
|
||||
font-size: 0.75em;
|
||||
}
|
||||
|
||||
div.jx-credit em {
|
||||
font-weight: bold;
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
|
||||
/* Animation */
|
||||
|
||||
div.jx-image.transition {
|
||||
transition: width .5s ease;
|
||||
}
|
||||
|
||||
div.jx-handle.transition {
|
||||
transition: left .5s ease;
|
||||
}
|
||||
|
||||
.vertical div.jx-image.transition {
|
||||
transition: height .5s ease;
|
||||
}
|
||||
|
||||
.vertical div.jx-handle.transition {
|
||||
transition: top .5s ease;
|
||||
}
|
||||
|
||||
/* Knight Lab Credit */
|
||||
a.jx-knightlab {
|
||||
background-color: #000; /* IE 8 */
|
||||
background-color: rgba(0,0,0,.25);
|
||||
bottom: 0;
|
||||
display: table;
|
||||
height: 14px;
|
||||
line-height: 14px;
|
||||
padding: 1px 4px 1px 5px;
|
||||
position: absolute;
|
||||
right: 0;
|
||||
text-decoration: none;
|
||||
z-index: 10;
|
||||
}
|
||||
|
||||
a.jx-knightlab div.knightlab-logo {
|
||||
display: inline-block;
|
||||
vertical-align: middle;
|
||||
height: 8px;
|
||||
width: 8px;
|
||||
background-color: #c34528;
|
||||
transform: rotate(45deg);
|
||||
-ms-transform: rotate(45deg);
|
||||
-webkit-transform: rotate(45deg);
|
||||
top: -1.25px;
|
||||
position: relative;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
a.jx-knightlab:hover {
|
||||
background-color: #000; /* IE 8 */
|
||||
background-color: rgba(0,0,0,.35);
|
||||
}
|
||||
a.jx-knightlab:hover div.knightlab-logo {
|
||||
background-color: #ce4d28;
|
||||
}
|
||||
|
||||
a.jx-knightlab span.juxtapose-name {
|
||||
display: table-cell;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
font-family: Helvetica, Arial, sans-serif;
|
||||
font-weight: 300;
|
||||
color: white;
|
||||
font-size: 10px;
|
||||
padding-left: 0.375em;
|
||||
vertical-align: middle;
|
||||
line-height: normal;
|
||||
text-shadow: none;
|
||||
}
|
||||
|
||||
/* keyboard accessibility */
|
||||
div.jx-controller:focus,
|
||||
div.jx-image.jx-left div.jx-label:focus,
|
||||
div.jx-image.jx-right div.jx-label:focus,
|
||||
a.jx-knightlab:focus {
|
||||
background: #eae34a;
|
||||
color: #000;
|
||||
}
|
||||
a.jx-knightlab:focus span.juxtapose-name{
|
||||
color: #000;
|
||||
border: none;
|
||||
}
|
||||
Vendored
+8
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user