Compare commits

..
Author SHA1 Message Date
shadowcz007 d55cbdfde5 Update App.py 2024-01-29 10:49:04 +08:00
shadowcz007 00cef5f37f test appnode 2024-01-28 23:25:46 +08:00
43 changed files with 1525 additions and 7164 deletions
+1 -3
View File
@@ -3,6 +3,4 @@ https/
nodes/config.json
workflow/my_workflow.json
workflow/my_workflow_app.json
workflow/prompt_result.json
app/*
workflow/prompt_result.json
app/*
+21 -59
View File
@@ -1,27 +1,13 @@
> 适配了最新版comfyui的py3.11 ,torch 2.1.2+cu121
> [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)
####
[comfyui-sd-prompt-mixlab](https://github.com/shadowcz007/comfyui-sd-prompt-mixlab)
[comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)
[comfyui-ultralytics-yolo](https://github.com/shadowcz007/comfyui-ultralytics-yolo)
[comfyui-moondream](https://github.com/shadowcz007/comfyui-moondream)
<!-- [comfyui-CLIPSeg](https://github.com/shadowcz007/comfyui-CLIPSeg) -->
> [discord](https://discord.gg/cXs9vZSqeK)
## 🚀🚗🚚🏃 Workflow-to-APP
- 新增AppInfo节点,可以通过简单的配置,把workflow转变为一个Web APP。
- 支持多个web app 切换
- 发布为app的workflow,可以在右键里再次编辑了
- web app可以设置分类,在comfyui右键菜单可以编辑更新web app
- 支持动态提示
![](./assets/微信图片_20240421205440.png)
- Support multiple web app switching.
- Add the AppInfo node, which allows you to transform the workflow into a web app by simple configuration.
@@ -45,19 +31,12 @@ APP-JSON:
- [image-to-image](./example/Image-to-Image_2.json)
- text-to-text
> 暂时支持 9 种节点作为界面上的输入节点:Load Image、VHS_LoadVideo、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
> 暂时支持8种节点作为界面上的输入节点:Load Image、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
> 输出节点:PreviewImage 、SaveImage、ShowTextForGPT、VHS_VideoCombine、PromptImage
> seed统一输入控件,支持:SamplerCustom、KSampler
> 配套[ps插件](https://github.com/shadowcz007/comfyui-ps-plugin)
> 如果遇到上传图片不成功,请检查下:局域网或者是云服务,请使用https,端口8189这个服务( 感谢 @Damien 反馈问题)
> If you encounter difficulties in uploading images, please check the following: for local network or cloud services, please use HTTPS and the service on port 8189. (Thanks to @Damien for reporting the issue.)
## 🏃🚗🚚🚀 Real-time Design
> ScreenShareNode & FloatingVideoNode. Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
@@ -79,7 +58,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
[Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
### GPT
> Support for calling multiple GPTs.ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
> Support for calling multiple GPTs.ChatGPT、ChatGLM3 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 . Azure OpenAI:https://xxxx.openai.azure.com
![gpt-workflow.svg](./assets/gpt-workflow.svg)
@@ -120,31 +99,18 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
[workflow](./workflow/3D-workflow.json)
### Image
#### LoadImagesToBatch
> Upload multiple images for batch input into the IP adapter.
#### LoadImagesFromLocal
### LoadImagesFromLocal
> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop.
![watch](./assets/4-loadfromlocal-watcher-workflow.svg)
[workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)
#### LoadImagesFromURL
### LoadImagesFromURL
> Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.
### Style
> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting)
![](./assets/VisualStylePrompting.png)
> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
### Utils
## Utils
> The Color node provides a color picker for easy color selection, the Font node offers built-in font selection for use with TextImage to generate text images, and the DynamicDelayByText node allows delayed execution based on the length of the input text.
- [添加了DynamicDelayByText功能,可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
@@ -157,7 +123,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
### Other Nodes
## Other Nodes
![main](./assets/all-workflow.svg)
![main2](./assets/detect-face-all.png)
@@ -171,6 +137,15 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
![TransparentImage](./assets/TransparentImage.png)
> Consistency Decoder
[openai Consistency Decoder]( https://github.com/openai/consistencydecoder)
![Consistency](./assets/consistency.png)
After downloading the OpenAI VAE model, place it in the "model/vae" directory for use.
https://openaipublic.azureedge.net/diff-vae/c9cebd3132dd9c42936d803e33424145a748843c8f716c0814838bdc8a2fe7cb/decoder.pt
> FeatheredMask、SmoothMask
Add edges to an image.
@@ -183,13 +158,6 @@ Add edges to an image.
from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainting)
> rembgNode
"briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"
*** briarmbg *** model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
### Improvement
- Add "help" option to the context menu for each node.
@@ -206,6 +174,8 @@ An improvement has been made to directly redirect to GitHub to search for missin
[Download rembg Models](https://github.com/danielgatis/rembg/tree/main#Models),move to:models/rembg
[Download CLIPSeg](https://huggingface.co/CIDAS/clipseg-rd64-refined/tree/main), move to : models/clipseg
[Download lama](https://github.com/enesmsahin/simple-lama-inpainting/releases/download/v0.1.0/big-lama.pt), move to : models/lama
[Download Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base), move to : models/clip_interrogator/Salesforce/blip-image-captioning-base
@@ -243,26 +213,18 @@ If you are using a venv, make sure you have it activated before installation and
pip3 install -r requirements.txt
```
#### Chinese community
访问 [www.mixcomfy.com](https://www.mixcomfy.com),获得更多内测功能,关注微信公众号:Mixlab无界社区
####
File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
#### Thanks:
[ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
#### discussions:
[discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
<picture>
<source
media="(prefers-color-scheme: dark)"
+34 -140
View File
@@ -6,7 +6,7 @@ import sys,json
import urllib
import hashlib
import datetime
import folder_paths
python = sys.executable
@@ -192,7 +192,7 @@ def read_workflow_json_files(folder_path ):
def get_workflows():
# print("#####path::", current_path)
workflow_path=os.path.join(current_path, "workflow")
# print('workflow_path: ',workflow_path)
# print('##workflow_path: ',workflow_path)
if not os.path.exists(workflow_path):
# 使用mkdir()方法创建新目录
os.mkdir(workflow_path)
@@ -222,8 +222,7 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
# print(item)
try:
x=item["data"]
# 管理员模式,读取全部数据
if i==0 or is_all:
if i==0:
apps.append({
"filename":item["filename"],
# "category":item['category'],
@@ -252,8 +251,7 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
"name":x['app']['name'],
"version":x['app']['version'],
"input":input,
"output":output,
"id":x['app']['id']
"output":output
}
},
"date":item["date"]
@@ -301,8 +299,7 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
"name":x['app']['name'],
"version":x['app']['version'],
"input":input,
"output":output,
"id":x['app']['id']
"output":output
}
},
"date":item["date"]
@@ -310,31 +307,6 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
return apps
# 历史记录
def save_prompt_result(id,data):
prompt_result_path=os.path.join(current_path, "workflow/prompt_result.json")
prompt_result={}
if os.path.exists(prompt_result_path):
with open(prompt_result_path) as json_file:
prompt_result = json.load(json_file)
prompt_result[id]=data
with open(prompt_result_path, 'w') as file:
json.dump(prompt_result, file)
return prompt_result_path
def get_prompt_result():
prompt_result_path=os.path.join(current_path, "workflow/prompt_result.json")
prompt_result={}
if os.path.exists(prompt_result_path):
with open(prompt_result_path) as json_file:
prompt_result = json.load(json_file)
res=list(prompt_result.values())
# print(res)
return res
def save_workflow_json(data):
workflow_path=os.path.join(current_path, "workflow/my_workflow.json")
with open(workflow_path, 'w') as file:
@@ -419,6 +391,9 @@ async def new_start(self, address, port, verbose=True, call_on_start=None):
site = web.TCPSite(runner, address, port)
await site.start()
PromptServer.instance.port=port
import ssl
crt, key = create_for_https()
ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
@@ -521,6 +496,7 @@ async def mixlab_workflow_hander(request):
'status':'success',
}
elif data['task']=='list':
# 暂时没有用到
result={
'data':get_workflows(),
'status':'success',
@@ -545,37 +521,6 @@ async def nodes_map_hander(request):
return web.json_response(result)
@routes.post("/mixlab/folder_paths")
async def get_checkpoints(request):
data = await request.json()
t="checkpoints"
try:
t=data['type']
except Exception as e:
print('/mixlab/folder_paths',False,e)
names = folder_paths.get_filename_list(t)
return web.json_response({"names":names,"types":list(folder_paths.folder_names_and_paths.keys())})
@routes.post("/mixlab/prompt_result")
async def post_prompt_result(request):
data = await request.json()
res=None
# print(data)
try:
action=data['action']
if action=='save':
result=data['data']
res=save_prompt_result(result['prompt_id'],result)
elif action=='all':
res=get_prompt_result()
except Exception as e:
print('/mixlab/prompt_result',False,e)
return web.json_response({"result":res})
# 扩展api接口
# from server import PromptServer
# from aiohttp import web
@@ -588,32 +533,26 @@ async def post_prompt_result(request):
# 导入节点
from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
from .nodes.ImageNode import LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
# from .nodes.Vae import VAELoader,VAEDecode
from .nodes.PromptNode import EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
from .nodes.ImageNode import SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
from .nodes.Vae import VAELoader,VAEDecode
from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText,TextSplitByDelimiter
from .nodes.Clipseg import CLIPSeg,CombineMasks
from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText
from .nodes.Audio import GamePal,SpeechRecognition,SpeechSynthesis
from .nodes.Utils import IncrementingListNode,ListSplit,CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,TESTNODE_TOKEN,AppInfo,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
from .nodes.Mask import PreviewMask_,MaskListReplace,MaskListMerge,OutlineMask,FeatheredMask
from .nodes.Style import ApplyVisualStylePrompting,StyleAlignedReferenceSampler,StyleAlignedBatchAlign,StyleAlignedSampleReferenceLatents
from .nodes.Video import VideoCombine_Adv,LoadVideoAndSegment,ImageListReplace,VAEEncodeForInpaint_Frames
from .nodes.Utils import CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
from .nodes.Mask import OutlineMask,FeatheredMask
from .nodes.App import AppInfo,AppNode
# 要导出的所有节点及其名称的字典
# 注意:名称应全局唯一
NODE_CLASS_MAPPINGS = {
"AppInfo":AppInfo,
"AppNode":AppNode,
"TESTNODE_":TESTNODE_,
"TESTNODE_TOKEN":TESTNODE_TOKEN,
"RandomPrompt":RandomPrompt,
# "LoraPrompt":LoraPrompt,
"EmbeddingPrompt":EmbeddingPrompt,
"PromptSlide":PromptSlide,
"GLIGENTextBoxApply_Advanced":GLIGENTextBoxApply_Advanced,
"PromptSimplification":PromptSimplification,
"PromptImage":PromptImage,
"MirroredImage":MirroredImage,
@@ -623,7 +562,6 @@ NODE_CLASS_MAPPINGS = {
"ResizeImageMixlab":ResizeImage,
"LoadImagesFromPath":LoadImagesFromPath,
"LoadImagesFromURL":LoadImagesFromURL,
"LoadImagesToBatch":LoadImages_,
"TextImage":TextImage,
"EnhanceImage":EnhanceImage,
"SvgImage":SvgImage,
@@ -631,12 +569,9 @@ NODE_CLASS_MAPPINGS = {
"ImageColorTransfer":ImageColorTransfer,
"ShowLayer":ShowLayer,
"NewLayer":NewLayer,
"CompositeImages_":CompositeImages,
"SplitImage":SplitImage,
"CenterImage":CenterImage,
"GridOutput":GridOutput,
"GridDisplayAndSave":GridDisplayAndSave,
"GridInput":GridInput,
"MergeLayers":MergeLayers,
"SplitLongMask":SplitLongMask,
"FeatheredMask":FeatheredMask,
@@ -644,17 +579,16 @@ NODE_CLASS_MAPPINGS = {
"FaceToMask":FaceToMask,
"AreaToMask":AreaToMask,
"ImageCropByAlpha":ImageCropByAlpha,
"ImagesPrompt_":ImagesPrompt,
# "VAELoaderConsistencyDecoder":VAELoader,
"VAELoaderConsistencyDecoder":VAELoader,
"SaveImageToLocal":SaveImageToLocal,
"SaveImageAndMetadata_":SaveImageAndMetadata,
# "VAEDecodeConsistencyDecoder":VAEDecode,
"VAEDecodeConsistencyDecoder":VAEDecode,
"ScreenShare":ScreenShareNode,
"FloatingVideo":FloatingVideo,
"CLIPSeg_":CLIPSeg,
"CombineMasks_":CombineMasks,
"ChatGPTOpenAI":ChatGPTNode,
"ShowTextForGPT":ShowTextForGPT,
"CharacterInText":CharacterInText,
"TextSplitByDelimiter":TextSplitByDelimiter,
"SpeechRecognition":SpeechRecognition,
"SpeechSynthesis":SpeechSynthesis,
"Color":ColorInput,
@@ -669,79 +603,39 @@ NODE_CLASS_MAPPINGS = {
"SwitchByIndex":SwitchByIndex,
"LimitNumber":LimitNumber,
"OutlineMask":OutlineMask,
"MaskListMerge_":MaskListMerge,
"JoinWithDelimiter":JoinWithDelimiter,
"Seed_":CreateSeedNode,
"CkptNames_":CreateCkptNames,
"SamplerNames_":CreateSampler_names,
"LoraNames_":CreateLoraNames,
"ApplyVisualStylePrompting_":ApplyVisualStylePrompting,
"StyleAlignedReferenceSampler_": StyleAlignedReferenceSampler,
"StyleAlignedSampleReferenceLatents_": StyleAlignedSampleReferenceLatents,
"StyleAlignedBatchAlign_": StyleAlignedBatchAlign,
"LoadVideoAndSegment_":LoadVideoAndSegment,
"VideoCombine_Adv":VideoCombine_Adv,
"ListSplit_":ListSplit,
"MaskListReplace_":MaskListReplace,
"ImageListReplace_":ImageListReplace,
"VAEEncodeForInpaint_Frames":VAEEncodeForInpaint_Frames,
"IncrementingListNode_":IncrementingListNode,
"PreviewMask_":PreviewMask_
"LoraNames_":CreateLoraNames
# "LaMaInpainting":LaMaInpainting
# "GamePal":GamePal
}
# 一个包含节点友好/可读的标题的字典
NODE_DISPLAY_NAME_MAPPINGS = {
"AppInfo":"App Info ♾️MixlabApp",
"Color":"Color Input ♾️MixlabApp",
"TextInput_":"Text Input ♾️MixlabApp",
"FloatSlider":"Float Slider Input ♾️MixlabApp",
"IntNumber":"Int Input ♾️MixlabApp",
"ImagesPrompt_":"Images Input ♾️MixlabApp",
"SaveImageAndMetadata_":"Save Image Output ♾️MixlabApp",
"ResizeImageMixlab":"Resize Image ♾️Mixlab",
"AppInfo":"AppInfo ♾️Mixlab",
"ResizeImageMixlab":"ResizeImage ♾️Mixlab",
"RandomPrompt": "Random Prompt ♾️Mixlab",
"PromptImage":"Output Prompt and Image ♾️Mixlab",
"SplitLongMask":"Splitting a long image into sections",
"VAELoaderConsistencyDecoder":"Consistency Decoder Loader",
"VAEDecodeConsistencyDecoder":"Consistency Decoder Decode",
"ScreenShare":"Screen Share ♾️Mixlab",
"ScreenShare":"ScreenShare ♾️Mixlab",
"FloatingVideo":"FloatingVideo ♾️Mixlab",
"ChatGPTOpenAI":"ChatGPT ♾️Mixlab",
"ShowTextForGPT":"Show Text ♾️MixlabApp",
"MergeLayers":"Merge Layers ♾️Mixlab",
"ShowTextForGPT":"ShowTextForGPT ♾️Mixlab",
"MergeLayers":"MergeLayers ♾️Mixlab",
"SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
"SpeechRecognition":"SpeechRecognition ♾️Mixlab",
"3DImage":"3DImage ♾️Mixlab",
"CompositeImages_":"Composite Images ♾️Mixlab",
"DynamicDelayProcessor":"DynamicDelayByText ♾️Mixlab",
"LaMaInpainting":"LaMaInpainting ♾️Mixlab",
"PromptSlide":"Prompt Slide ♾️Mixlab",
"PromptGenerate_Mix":"Prompt Generate ♾️Mixlab",
"ChinesePrompt_Mix":"Chinese Prompt ♾️Mixlab",
"PromptSlide":"PromptSlide ♾️Mixlab",
"PromptGenerate_Mix":"PromptGenerate ♾️Mixlab",
"ChinesePrompt_Mix":"ChinesePrompt ♾️Mixlab",
"GamePal":"GamePal ♾️Mixlab",
"RembgNode_Mix":"Remove Background ♾️Mixlab",
"LoraNames_":"LoraName ♾️Mixlab",
"ApplyVisualStylePrompting_":"Apply VisualStyle Prompting ♾️Mixlab",
"StyleAlignedReferenceSampler_": "StyleAligned Reference Sampler ♾️Mixlab",
"StyleAlignedSampleReferenceLatents_": "StyleAligned Sample Reference Latents ♾️Mixlab",
"StyleAlignedBatchAlign_": "StyleAligned Batch Align ♾️Mixlab",
"LoadVideoAndSegment_":"Load Video And Segment ♾️Mixlab",
"VideoCombine_Adv":"Video Combine ♾️Mixlab",
"MaskListMerge_":"MaskList to Mask ♾️Mixlab",
"ListSplit_":"Split List ♾️Mixlab",
"MaskListReplace_":"MaskList Replace ♾️Mixlab",
"ImageListReplace_":"ImageList Replace ♾️Mixlab",
"SwitchByIndex":"List Switch By Index ♾️Mixlab",
"GLIGENTextBoxApply_Advanced":"GLIGEN TextBox Apply ♾️Mixlab",
"GridDisplayAndSave":"Grid Display And Save ♾️Mixlab",
"GridInput":"Grid Input ♾️Mixlab",
"GridOutput":"Grid Output ♾️Mixlab",
"GetImageSize_":"Get Image Size ♾️Mixlab",
"VAEEncodeForInpaint_Frames":"VAE Encode For Inpaint Frames ♾️Mixlab",
"IncrementingListNode_":"Create Incrementing Number List ♾️Mixlab",
"LoadImagesToBatch":"Load Images(base64) ♾️Mixlab",
"PreviewMask_":"Preview Mask"
"RembgNode_Mix":"Removebg",
"LoraNames_":"LoraName_TriggerWords.safetensors"
}
# web ui的节点功能
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 784 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 965 KiB

+5
View File
@@ -4768,6 +4768,7 @@
"ChinesePrompt_Mix",
"3DImage",
"AppInfo",
"AppNode",
"IntNumber",
"FloatSlider",
"ResizeImage",
@@ -4775,6 +4776,7 @@
"PromptImage",
"SaveImageToLocal",
"AreaToMask",
"CLIPSeg_",
"CharacterInText",
"ChatGPTOpenAI",
"Color",
@@ -4782,6 +4784,7 @@
"CkptNames_",
"SamplerNames_",
"LoraNames_",
"CombineMasks_",
"EnhanceImage",
"GradientImage",
"FaceToMask",
@@ -4809,6 +4812,8 @@
"TextImage",
"ResizeImageMixlab",
"TransparentImage",
"VAEDecodeConsistencyDecoder",
"VAELoaderConsistencyDecoder",
"TextToNumber",
"TextInput_",
"DynamicDelayProcessor",
-10
View File
@@ -1,10 +0,0 @@
[
{
"keyword":"Dog",
"imgurl":"http://127.0.0.1:8188/view?filename=1709966910233.png&type=input&subfolder=&rand=0.2734446552394221"
},
{
"keyword":"x",
"imgurl":"http://127.0.0.1:8188/view?filename=image%20(33).png&type=input&subfolder=pasted&rand=0.6984318219852814"
}
]
-1
View File
@@ -1 +0,0 @@
{}
+397
View File
@@ -0,0 +1,397 @@
import os,sys
# import re,random,json
from PIL import Image
import numpy as np
# FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
import folder_paths
#
import os
import json
import datetime
import folder_paths
from server import PromptServer
import importlib.util
def is_installed(package):
try:
spec = importlib.util.find_spec(package)
except ModuleNotFoundError:
return False
return spec is not None
class AnyType(str):
"""A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
def __ne__(self, __value: object) -> bool:
return False
any_type = AnyType("*")
app_path = os.path.abspath(os.path.join(os.path.dirname(__file__),'../app'))
# workflow 目录下的所有json
def read_workflow_json_files_all(folder_path):
print('#read_workflow_json_files_all',folder_path)
json_files = []
for root, dirs, files in os.walk(folder_path):
for file in files:
if file.endswith('.json'):
json_files.append(os.path.join(root, file))
data = []
for file_path in json_files:
try:
with open(file_path) as json_file:
json_data = json.load(json_file)
creation_time = datetime.datetime.fromtimestamp(os.path.getctime(file_path))
numeric_timestamp = creation_time.timestamp()
option=os.path.basename(os.path.dirname(file_path))+'/'+os.path.basename(file_path)
if os.path.dirname(file_path) == folder_path:
option=os.path.basename(file_path)
file_info = {
'filename': os.path.basename(file_path),
'category': os.path.dirname(file_path),
'data': json_data,
'date': numeric_timestamp,
"option":option
}
data.append(file_info)
except Exception as e:
print(e)
sorted_data = sorted(data, key=lambda x: x['date'], reverse=True)
return sorted_data
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
def create_temp_file(image):
output_dir = folder_paths.get_temp_directory()
(
full_output_folder,
filename,
counter,
subfolder,
_,
) = folder_paths.get_save_image_path('tmp', output_dir)
im=tensor2pil(image)
image_file = f"{filename}_{counter:05}.png"
image_path=os.path.join(full_output_folder, image_file)
im.save(image_path,compress_level=4)
return [{
"filename": image_file,
"subfolder": subfolder,
"type": "temp"
}]
try:
if is_installed('websocket')==False:
import subprocess
# 安装
print('#pip install websocket-client')
result = subprocess.run([sys.executable, '-s', '-m', 'pip', 'install', 'websocket-client'], capture_output=True, text=True)
#检查命令执行结果
if result.returncode == 0:
print("#install success")
import websocket
else:
print("#install error")
else:
import websocket
# NOTE: websocket-client (https://github.com/websocket-client/websocket-client)
except:
print("#websocket-client error")
#This is an example that uses the websockets api to know when a prompt execution is done
#Once the prompt execution is done it downloads the images using the /history endpoint
import uuid
import json
import urllib.request
import urllib.parse
server_address = "127.0.0.1:8188"
client_id = str(uuid.uuid4())
def queue_prompt(prompt):
p = {"prompt": prompt, "client_id": client_id}
data = json.dumps(p).encode('utf-8')
req = urllib.request.Request("http://{}/prompt".format(server_address), data=data)
return json.loads(urllib.request.urlopen(req).read())
def get_image(filename, subfolder, folder_type):
data = {"filename": filename, "subfolder": subfolder, "type": folder_type}
url_values = urllib.parse.urlencode(data)
with urllib.request.urlopen("http://{}/view?{}".format(server_address, url_values)) as response:
return response.read()
def get_history(prompt_id):
with urllib.request.urlopen("http://{}/history/{}".format(server_address, prompt_id)) as response:
return json.loads(response.read())
def get_images(ws, prompt):
prompt_id = queue_prompt(prompt)['prompt_id']
output_images = {}
while True:
out = ws.recv()
if isinstance(out, str):
message = json.loads(out)
if message['type'] == 'executing':
data = message['data']
if data['node'] is None and data['prompt_id'] == prompt_id:
break #Execution is done
else:
continue #previews are binary data
history = get_history(prompt_id)[prompt_id]
for o in history['outputs']:
for node_id in history['outputs']:
node_output = history['outputs'][node_id]
if 'images' in node_output:
images_output = []
for image in node_output['images']:
image_data = get_image(image['filename'], image['subfolder'], image['type'])
images_output.append(image_data)
output_images[node_id] = images_output
return output_images
prompt_text = """
{
"3": {
"class_type": "KSampler",
"inputs": {
"cfg": 8,
"denoise": 1,
"latent_image": [
"5",
0
],
"model": [
"4",
0
],
"negative": [
"7",
0
],
"positive": [
"6",
0
],
"sampler_name": "euler",
"scheduler": "normal",
"seed": 8566257,
"steps": 20
}
},
"4": {
"class_type": "CheckpointLoaderSimple",
"inputs": {
"ckpt_name": "v1-5-pruned-emaonly.ckpt"
}
},
"5": {
"class_type": "EmptyLatentImage",
"inputs": {
"batch_size": 1,
"height": 512,
"width": 512
}
},
"6": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"4",
1
],
"text": "masterpiece best quality girl"
}
},
"7": {
"class_type": "CLIPTextEncode",
"inputs": {
"clip": [
"4",
1
],
"text": "bad hands"
}
},
"8": {
"class_type": "VAEDecode",
"inputs": {
"samples": [
"3",
0
],
"vae": [
"4",
2
]
}
},
"9": {
"class_type": "SaveImage",
"inputs": {
"filename_prefix": "ComfyUI",
"images": [
"8",
0
]
}
}
}
"""
# prompt = json.loads(prompt_text)
# #set the text prompt for our positive CLIPTextEncode
# prompt["6"]["inputs"]["text"] = "masterpiece best quality man"
# #set the seed for our KSampler node
# prompt["3"]["inputs"]["seed"] = 5
# ws = websocket.WebSocket()
# ws.connect("ws://{}/ws?clientId={}".format(server_address, client_id))
# images = get_images(ws, prompt)
# #Commented out code to display the output images:
# # for node_id in images:
# # for image_data in images[node_id]:
# # from PIL import Image
# # import io
# # image = Image.open(io.BytesIO(image_data))
# # image.show()
# app 配置节点
class AppInfo:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"name": ("STRING",{"multiline": False,"default": "Mixlab-App","dynamicPrompts": False}),
"input_ids":("STRING",{"multiline": True,"default": "\n".join(["1","2","3"]),"dynamicPrompts": False}),
"output_ids":("STRING",{"multiline": True,"default": "\n".join(["5","9"]),"dynamicPrompts": False}),
},
"optional":{
"IMAGE": ("IMAGE",),
"description":("STRING",{"multiline": True,"default": "","dynamicPrompts": False}),
"version":("INT", {
"default": 1,
"min": 1,
"max": 10000,
"step": 1,
"display": "number"
}),
"share_prefix":("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
"link":("STRING",{"multiline": False,"default": "https://","dynamicPrompts": False}),
"category":("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
"auto_save": (["enable","disable"],),
}
}
RETURN_TYPES = ()
# RETURN_NAMES = ("IMAGE",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab"
OUTPUT_NODE = True
INPUT_IS_LIST = True
# OUTPUT_IS_LIST = (True,)
def run(self,name,input_ids,output_ids,IMAGE,description,version,share_prefix,link,category,auto_save):
name=name[0]
im=None
if IMAGE:
im=IMAGE[0][0]
#TODO batch 的方式需要处理
im=create_temp_file(im)
# image [img,] img[batch,w,h,a] 列表里面是batch,
input_ids=input_ids[0]
output_ids=output_ids[0]
description=description[0]
version=version[0]
share_prefix=share_prefix[0]
link=link[0]
category=category[0]
# id=get_json_hash([name,im,input_ids,output_ids,description,version])
return {"ui": {"json": [name,im,input_ids,output_ids,description,version,share_prefix,link,category]}, "result": ()}
# app可以当成节点运行
class AppNode:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"name": ([x['option'] for x in read_workflow_json_files_all(app_path)],),
"image":("IMAGE",),
}
}
RETURN_TYPES = (any_type,)
RETURN_NAMES = ("output",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab"
OUTPUT_NODE = True
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
def run(self,name,input):
print('#app_path',input)
print(PromptServer.instance.port)
return (name,)
+16 -172
View File
@@ -3,35 +3,13 @@ import time
import urllib.error
import re,json,os,string,random
import folder_paths
import hashlib
import codecs,sys
import importlib.util
def is_installed(package):
try:
spec = importlib.util.find_spec(package)
except ModuleNotFoundError:
return False
return spec is not None
def get_unique_hash(string):
hash_object = hashlib.sha1(string.encode())
unique_hash = hash_object.hexdigest()
return unique_hash
def generate_random_string(length):
letters = string.ascii_letters + string.digits
return ''.join(random.choice(letters) for _ in range(length))
class AnyType(str):
"""A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
def __ne__(self, __value: object) -> bool:
return False
any_type = AnyType("*")
# 判断是否是azure服务
def is_azure_url(url):
@@ -58,33 +36,6 @@ def openai_client(key,url):
)
return client
def ZhipuAI_client(key):
try:
if is_installed('zhipuai')==False:
import subprocess
# 安装
print('#pip install zhipuai')
result = subprocess.run([sys.executable, '-s', '-m', 'pip', 'install', 'zhipuai'], capture_output=True, text=True)
#检查命令执行结果
if result.returncode == 0:
print("#install success")
from zhipuai import ZhipuAI
else:
print("#install error")
else:
from zhipuai import ZhipuAI
except:
print("#install zhipuai error")
client = ZhipuAI(
api_key=key, # 填写您的 APIKey
)
return client
def chat(client, model_name,messages ):
@@ -138,15 +89,7 @@ class ChatGPTNode:
"default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
"multiline": True,"dynamicPrompts": False
}),
"model": ([
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-35-turbo",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-4-0613",
"gpt-4-1106-preview",
"glm-4"],
"model": (["gpt-3.5-turbo","gpt-35-turbo","gpt-3.5-turbo-16k", "gpt-3.5-turbo-16k-0613", "gpt-4-0613","gpt-4-1106-preview"],
{"default": "gpt-3.5-turbo"}),
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "step": 1}),
"context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
@@ -189,13 +132,8 @@ class ChatGPTNode:
if is_azure_url(api_url):
client=azure_client(api_key,api_url)
else:
# 根据用户选择的模型,设置相应的接口和模型名称
if model == "glm-4" :
client = ZhipuAI_client(api_key) # 使用 Zhipuai 的接口
print('using Zhipuai interface')
else :
client = openai_client(api_key,api_url) # 使用 ChatGPT 的接口
print('using ChatGPT interface')
client=openai_client(api_key,api_url)
print('openai url')
# 把用户的提示添加到会话历史中
# 调用API时传递整个会话历史
@@ -240,7 +178,7 @@ class ShowTextForGPT:
"text": ("STRING", {"forceInput": True,"dynamicPrompts": False}),
},
"optional":{
"output_dir": ("STRING",{"forceInput": True,"default": "","multiline": True,"dynamicPrompts": False}),
"output_dir": ("STRING",{"default": "","multiline": True,"dynamicPrompts": False}),
}
}
@@ -250,62 +188,18 @@ class ShowTextForGPT:
OUTPUT_NODE = True
OUTPUT_IS_LIST = (True,)
CATEGORY = "♾️Mixlab/Text"
CATEGORY = "♾️Mixlab/GPT"
def run(self, text,output_dir=[""]):
def run(self, text,output_dir):
output_dir=output_dir[0]
filename=generate_random_string(4)+'.txt'
if output_dir=='':
output_dir = folder_paths.get_temp_directory()
# 类型纠正
texts=[]
for t in text:
if not isinstance(t, str):
t = str(t)
texts.append(t)
text=texts
if len(output_dir)==1 and (output_dir[0]=='' or os.path.dirname(output_dir[0])==''):
t='\n'.join(text)
output_dir=[
os.path.join(folder_paths.get_temp_directory(),
get_unique_hash(t)+'.txt'
)
]
elif len(output_dir)==1:
base=os.path.basename(output_dir[0])
t='\n'.join(text)
if base=='' or os.path.splitext(base)[1]=='':
base=get_unique_hash(t)+'.txt'
output_dir=[
os.path.join(output_dir[0],
base
)
]
# elif len(output_dir)>1:
if len(output_dir)==1 and len(text)>1:
output_dir=[output_dir[0] for _ in range(len(text))]
for i in range(len(text)):
o_fp=output_dir[i]
dirp=os.path.dirname(o_fp)
if dirp=='':
dirp=folder_paths.get_temp_directory()
o_fp=os.path.join(folder_paths.get_temp_directory(),o_fp
)
if not os.path.exists(dirp):
os.mkdir(dirp)
if not os.path.splitext(o_fp)[1].lower()=='.txt':
o_fp=o_fp+'.txt'
t=text[i]
with open(o_fp, 'w') as file:
file.write(t)
save_to_dirpath=os.path.join(output_dir,filename)
with open(save_to_dirpath, 'w') as file:
file.write("\n".join(text))
# print(text)
return {"ui": {"text": text}, "result": (text,)}
@@ -334,61 +228,11 @@ class CharacterInText:
# OUTPUT_NODE = True
OUTPUT_IS_LIST = (False,)
CATEGORY = "♾️Mixlab/Text"
CATEGORY = "♾️Mixlab/GPT"
def run(self, text,character,start_index):
# print(text,character,start_index)
b=1 if character.lower() in text.lower() else 0
b=1 if character in text else 0
return (b+start_index,)
class TextSplitByDelimiter:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"text": ("STRING", {"multiline": True,"dynamicPrompts": False}),
"delimiter":("STRING", {"multiline": False,"default":",","dynamicPrompts": False}),
"start_index": ("INT", {
"default": 0,
"min": 0, #Minimum value
"max": 1000, #Maximum value
"step": 1, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"skip_every": ("INT", {
"default": 0,
"min": 0, #Minimum value
"max": 10, #Maximum value
"step": 1, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"max_count": ("INT", {
"default": 10,
"min": 1, #Minimum value
"max": 1000, #Maximum value
"step": 1, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
}
}
INPUT_IS_LIST = False
RETURN_TYPES = ("STRING",)
FUNCTION = "run"
# OUTPUT_NODE = True
OUTPUT_IS_LIST = (True,)
CATEGORY = "♾️Mixlab/Text"
def run(self, text,delimiter,start_index,skip_every,max_count):
if delimiter=="":
arr=[text.strip()]
else:
delimiter=codecs.decode(delimiter, 'unicode_escape')
arr= [line for line in text.split(delimiter) if line.strip()]
arr= arr[start_index:start_index + max_count * (skip_every+1):(skip_every+1)]
return (arr,)
+2 -1
View File
@@ -77,7 +77,8 @@ if not os.path.exists(caption_model_path):
caption_model_path='Salesforce/blip-image-captioning-base'
cache_path=os.path.join(folder_paths.models_dir, "clip_interrogator")
if not os.path.exists(cache_path):
os.mkdir(cache_path)
# Tensor to PIL
def tensor2pil(image):
+272
View File
@@ -0,0 +1,272 @@
#### Thanks:
# [ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation
from PIL import Image
import torch
import torchvision.transforms as T
import numpy as np
from torchvision.transforms.functional import to_pil_image
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import cv2
from scipy.ndimage import gaussian_filter
from typing import Optional, Tuple
import warnings,os
warnings.filterwarnings("ignore", category=UserWarning, module="torch")
warnings.filterwarnings("ignore", category=UserWarning, module="safetensors")
import folder_paths
import logging
logger = logging.getLogger('CLIPSeg nodes')
clipseg_model_dir = os.path.join(folder_paths.models_dir, "clipseg")
if not os.path.exists(clipseg_model_dir):
print(f"## clipseg model not found: {clipseg_model_dir},pls download from https://huggingface.co/CIDAS/clipseg-rd64-refined/tree/main")
clipseg_model_dir='CIDAS/clipseg-rd64-refined'
"""Helper methods for CLIPSeg nodes"""
# Tensor to PIL
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
# Convert PIL to Tensor
def pil2tensor(image):
return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
def tensor_to_numpy(tensor: torch.Tensor) -> np.ndarray:
"""Convert a tensor to a numpy array and scale its values to 0-255."""
array = tensor.numpy().squeeze()
return (array * 255).astype(np.uint8)
def numpy_to_tensor(array: np.ndarray) -> torch.Tensor:
"""Convert a numpy array to a tensor and scale its values from 0-255 to 0-1."""
array = array.astype(np.float32) / 255.0
return torch.from_numpy(array)[None,]
def apply_colormap(mask: torch.Tensor, colormap) -> np.ndarray:
"""Apply a colormap to a tensor and convert it to a numpy array."""
colored_mask = colormap(mask.numpy())[:, :, :3]
return (colored_mask * 255).astype(np.uint8)
def resize_image(image: np.ndarray, dimensions: Tuple[int, int]) -> np.ndarray:
"""Resize an image to the given dimensions using linear interpolation."""
return cv2.resize(image, dimensions, interpolation=cv2.INTER_LINEAR)
def overlay_image(background: np.ndarray, foreground: np.ndarray, alpha: float) -> np.ndarray:
"""Overlay the foreground image onto the background with a given opacity (alpha)."""
return cv2.addWeighted(background, 1 - alpha, foreground, alpha, 0)
def dilate_mask(mask: torch.Tensor, dilation_factor: float) -> torch.Tensor:
"""Dilate a mask using a square kernel with a given dilation factor."""
kernel_size = int(dilation_factor * 2) + 1
kernel = np.ones((kernel_size, kernel_size), np.uint8)
mask_dilated = cv2.dilate(mask.numpy(), kernel, iterations=1)
return torch.from_numpy(mask_dilated)
class CLIPSeg:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
"""
Return a dictionary which contains config for all input fields.
Some types (string): "MODEL", "VAE", "CLIP", "CONDITIONING", "LATENT", "IMAGE", "INT", "STRING", "FLOAT".
Input types "INT", "STRING" or "FLOAT" are special values for fields on the node.
The type can be a list for selection.
Returns: `dict`:
- Key input_fields_group (`string`): Can be either required, hidden or optional. A node class must have property `required`
- Value input_fields (`dict`): Contains input fields config:
* Key field_name (`string`): Name of a entry-point method's argument
* Value field_config (`tuple`):
+ First value is a string indicate the type of field or a list for selection.
+ Secound value is a config for type "INT", "STRING" or "FLOAT".
"""
return {"required":
{
"image": ("IMAGE",),
"text": ("STRING", {"multiline": False,"dynamicPrompts": False}),
},
"optional":
{
"blur": ("FLOAT", {"min": 0, "max": 15, "step": 0.1, "default": 3}),
"threshold": ("FLOAT", {"min": 0, "max": 1, "step": 0.05, "default": 0.3}),
"dilation_factor": ("INT", {"min": 0, "max": 10, "step": 1, "default": 4}),
}
}
CATEGORY = "♾️Mixlab/Mask"
RETURN_TYPES = ("MASK", "IMAGE", "IMAGE",)
RETURN_NAMES = ("Mask","Heatmap Mask", "BW Mask")
# INPUT_IS_LIST = True
OUTPUT_IS_LIST = (False,False,False,)
FUNCTION = "segment_image"
def segment_image(self, image: torch.Tensor, text: str, blur: float, threshold: float, dilation_factor: int) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""Create a segmentation mask from an image and a text prompt using CLIPSeg.
Args:
image (torch.Tensor): The image to segment.
text (str): The text prompt to use for segmentation.
blur (float): How much to blur the segmentation mask.
threshold (float): The threshold to use for binarizing the segmentation mask.
dilation_factor (int): How much to dilate the segmentation mask.
Returns:
Tuple[torch.Tensor, torch.Tensor, torch.Tensor]: The segmentation mask, the heatmap mask, and the binarized mask.
"""
# Convert the Tensor to a PIL image
image_np = image.numpy().squeeze() # Remove the first dimension (batch size of 1)
# Convert the numpy array back to the original range (0-255) and data type (uint8)
image_np = (image_np * 255).astype(np.uint8)
# Create a PIL image from the numpy array
i = Image.fromarray(image_np, mode="RGB")
processor = CLIPSegProcessor.from_pretrained(clipseg_model_dir)
model = CLIPSegForImageSegmentation.from_pretrained(clipseg_model_dir)
prompt = text
input_prc = processor(text=prompt, images=i, padding="max_length", return_tensors="pt")
# Predict the segemntation mask
with torch.no_grad():
outputs = model(**input_prc)
tensor = torch.sigmoid(outputs[0]) # get the mask
# Apply a threshold to the original tensor to cut off low values
thresh = threshold
tensor_thresholded = torch.where(tensor > thresh, tensor, torch.tensor(0, dtype=torch.float))
# Apply Gaussian blur to the thresholded tensor
sigma = blur
tensor_smoothed = gaussian_filter(tensor_thresholded.numpy(), sigma=sigma)
tensor_smoothed = torch.from_numpy(tensor_smoothed)
# Normalize the smoothed tensor to [0, 1]
mask_normalized = (tensor_smoothed - tensor_smoothed.min()) / (tensor_smoothed.max() - tensor_smoothed.min())
# Dilate the normalized mask
mask_dilated = dilate_mask(mask_normalized, dilation_factor)
# Convert the mask to a heatmap and a binary mask
heatmap = apply_colormap(mask_dilated, cm.viridis)
binary_mask = apply_colormap(mask_dilated, cm.Greys_r)
# Overlay the heatmap and binary mask on the original image
dimensions = (image_np.shape[1], image_np.shape[0])
heatmap_resized = resize_image(heatmap, dimensions)
binary_mask_resized = resize_image(binary_mask, dimensions)
alpha_heatmap, alpha_binary = 0.5, 1
overlay_heatmap = overlay_image(image_np, heatmap_resized, alpha_heatmap)
overlay_binary = overlay_image(image_np, binary_mask_resized, alpha_binary)
# Convert the numpy arrays to tensors
image_out_heatmap = numpy_to_tensor(overlay_heatmap)
image_out_binary = numpy_to_tensor(overlay_binary)
# Save or display the resulting binary mask
binary_mask_image = Image.fromarray(binary_mask_resized[..., 0])
# convert PIL image to numpy array
tensor_bw = binary_mask_image.convert("L")
tensor_bw=pil2tensor(tensor_bw)
# tensor_bw = np.array(tensor_bw).astype(np.float32) / 255.0
# tensor_bw = torch.from_numpy(tensor_bw)[None,]
# tensor_bw = tensor_bw.squeeze(0)[..., 0]
return (tensor_bw, image_out_heatmap, image_out_binary,)
#OUTPUT_NODE = False
class CombineMasks:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {"required":
{
"input_image": ("IMAGE", ),
"mask_1": ("MASK", ),
"mask_2": ("MASK", ),
},
"optional":
{
"mask_3": ("MASK",),
},
}
CATEGORY = "♾️Mixlab/Mask"
RETURN_TYPES = ("MASK", "IMAGE", "IMAGE",)
RETURN_NAMES = ("Combined Mask","Heatmap Mask", "BW Mask")
FUNCTION = "combine_masks"
def combine_masks(self, input_image: torch.Tensor, mask_1: torch.Tensor, mask_2: torch.Tensor, mask_3: Optional[torch.Tensor] = None) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""A method that combines two or three masks into one mask. Takes in tensors and returns the mask as a tensor, as well as the heatmap and binary mask as tensors."""
# Combine masks
if mask_1 is not None:
mask_1 = mask_1.squeeze()
if mask_2 is not None:
mask_2 = mask_2.squeeze()
if mask_3 is not None:
mask_3 = mask_3.squeeze()
print(mask_1.shape,mask_2.shape , mask_3.shape)
combined_mask = mask_1 + mask_2 + mask_3 if mask_3 is not None else mask_1 + mask_2
# print(combined_mask)
# Convert image and masks to numpy arrays
image_np = tensor_to_numpy(input_image)
heatmap = apply_colormap(combined_mask, cm.viridis)
binary_mask = apply_colormap(combined_mask, cm.Greys_r)
# Resize heatmap and binary mask to match the original image dimensions
dimensions = (image_np.shape[1], image_np.shape[0])
# print('heatmap',heatmap)
if dimensions is None or dimensions[0] == 0 or dimensions[1] == 0:
raise ValueError("Invalid dimensions")
heatmap_resized = resize_image(heatmap, dimensions)
binary_mask_resized = resize_image(binary_mask, dimensions)
# Overlay the heatmap and binary mask onto the original image
alpha_heatmap, alpha_binary = 0.5, 1
overlay_heatmap = overlay_image(image_np, heatmap_resized, alpha_heatmap)
overlay_binary = overlay_image(image_np, binary_mask_resized, alpha_binary)
# Convert overlays to tensors
image_out_heatmap = numpy_to_tensor(overlay_heatmap)
image_out_binary = numpy_to_tensor(overlay_binary)
return combined_mask, image_out_heatmap, image_out_binary
# A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique
# NODE_CLASS_MAPPINGS = {
# "CLIPSeg": CLIPSeg,
# "CombineSegMasks": CombineMasks,
# }
+58 -647
View File
@@ -8,90 +8,16 @@ import base64,os,random
from io import BytesIO
import folder_paths
import json,io
import comfy.utils
from comfy.cli_args import args
import cv2
import string
import math,glob
from .Watcher import FolderWatcher
import hashlib
# 将PIL图片转换为OpenCV格式
def pil_to_opencv(image):
open_cv_image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
return open_cv_image
# 将OpenCV格式图片转换为PIL格式
def opencv_to_pil(image):
pil_image = Image.fromarray(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
return pil_image
def composite_images(foreground, background, mask):
width,height=foreground.size
bg_image=background
# 按z-index排序
layer = {
"x":0,
"y":0,
"width":width,
"height":height,
"z_index":88,
"scale_option":'overall',
"image":foreground,
"mask":mask
}
width, height = bg_image.size
layer_image=layer['image']
layer_mask=layer['mask']
bg_image=merge_images(bg_image,
layer_image,
layer_mask,
layer['x'],
layer['y'],
layer['width'],
layer['height'],
layer['scale_option']
)
bg_image=bg_image.convert('RGB')
return bg_image
def count_files_in_directory(directory):
file_count = 0
for _, _, files in os.walk(directory):
file_count += len(files)
return file_count
def save_json_to_file(data, file_path):
with open(file_path, 'w') as file:
json.dump(data, file)
def draw_rectangle(image, grid, color,width):
x, y, w, h = grid
draw = ImageDraw.Draw(image)
draw.rectangle([(x, y), (x+w, y+h)], outline=color,width=width)
def generate_random_string(length):
letters = string.ascii_letters + string.digits
return ''.join(random.choice(letters) for _ in range(length))
def padding_rectangle(grid, padding):
x, y, w, h = grid
x -= padding
y -= padding
w += 2 * padding
h += 2 * padding
return (x, y, w, h)
class AnyType(str):
"""A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
@@ -454,7 +380,6 @@ def get_images_filepath(f,white_bg=False):
for root, dirs, files in os.walk(f):
for file in files:
file_path = os.path.join(root, file)
file_name=os.path.basename(file_path)
try:
imgs=load_image(file_path,white_bg)
for img in imgs:
@@ -462,7 +387,6 @@ def get_images_filepath(f,white_bg=False):
"image":img['image'],
"mask":img['mask'],
"file_path":file_path,
"file_name":file_name,
"psd":len(imgs)>1
})
except:
@@ -470,15 +394,12 @@ def get_images_filepath(f,white_bg=False):
elif os.path.isfile(f):
try:
file_path = os.path.join(root, f)
file_name=os.path.basename(file_path)
imgs=load_image(f,white_bg)
for img in imgs:
images.append({
"image":img['image'],
"mask":img['mask'],
"file_path":file_path,
"file_name":file_name,
"psd":len(imgs)>1
})
except:
@@ -633,7 +554,7 @@ def detect_faces(image):
def areaToMask(x,y,w,h,image):
# 创建一个与原图片大小相同的空白图片
mask = Image.new('L', image.size)
mask = Image.new('1', image.size)
# 创建一个可用于绘制的对象
draw = ImageDraw.Draw(mask)
@@ -666,46 +587,7 @@ def areaToMask(x,y,w,h,image):
# # bg_image.save("output.jpg")
# return bg_image
import cv2
import numpy as np
# ps的正片叠底
# 可以基于https://www.cnblogs.com/jsxyhelu/p/16947810.html ,用gpt写python代码
def multiply_blend(image1, image2):
image1=pil_to_opencv(image1)
image2=pil_to_opencv(image2)
# 将图像转换为浮点型
image1 = image1.astype(float)
image2 = image2.astype(float)
if image1.shape != image2.shape:
image1 = cv2.resize(image1, (image2.shape[1], image2.shape[0]))
# 归一化图像
image1 /= 255.0
image2 /= 255.0
# 正片叠底混合
blended = image1 * image2
# 将图像还原为8位无符号整数
blended = (blended * 255).astype(np.uint8)
blended=opencv_to_pil(blended)
return blended
# # 读取图像
# image1 = cv2.imread('1.png')
# image2 = cv2.imread('3.png')
# # 进行正片叠底混合
# result = multiply_blend(image1, image2)
# cv2.imwrite('result.jpg', result)
def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,is_multiply_blend=False):
def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option):
# 打开底图
bg_image = bg_image.convert("RGBA")
@@ -734,43 +616,13 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option,
nw, nh = layer_image.size
mask = mask.resize((nw, nh))
# 分离出a通道
r, g, b, alpha = layer_image.split()
alpha = ImageOps.invert(alpha)
# 创建一个新的RGB图像
new_rgb_image = Image.new("RGB", layer_image.size)
# 将透明通道粘贴到新的RGB图像上
new_rgb_image.paste(layer_image, (0, 0), mask=alpha)
new_rgb_image.paste(layer_image, (x, y), mask=mask)
mask=new_rgb_image.convert('L')
mask = ImageOps.invert(mask)
if is_multiply_blend:
bg_image_white=Image.new("RGB", bg_image.size,(255, 255, 255))
bg_image_white.paste(layer_image, (x, y), mask=mask)
bg_image=multiply_blend(bg_image_white,bg_image)
bg_image=bg_image.convert("RGBA")
else:
# 在底图上粘贴图层
bg_image.paste(layer_image, (x, y), mask=mask)
# 在底图上粘贴图层
bg_image.paste(layer_image, (x, y), mask=mask)
# 输出合成后的图片
return bg_image
def resize_2(img):
# 检查图像的高度是否是2的倍数,如果不是,则调整高度
if img.height % 2 != 0:
img = img.resize((img.width, img.height + 1))
# 检查图像的宽度是否是2的倍数,如果不是,则调整宽度
if img.width % 2 != 0:
img = img.resize((img.width + 1, img.height))
return img
# TODO 几个像素点的底
def resize_image(layer_image, scale_option, width, height,color="white"):
layer_image = layer_image.convert("RGB")
@@ -800,10 +652,8 @@ def resize_image(layer_image, scale_option, width, height,color="white"):
resized_image = Image.new("RGB", (width, height), color=color)
resized_image.paste(layer_image.resize((new_width, new_height)), ((width - new_width) // 2, (height - new_height) // 2))
resized_image = resized_image.convert("RGB")
resized_image=resize_2(resized_image)
return resized_image
layer_image=resize_2(layer_image)
return layer_image
@@ -1127,35 +977,6 @@ class TransparentImage:
class ImagesPrompt:
@classmethod
def INPUT_TYPES(s):
# input_dir = folder_paths.get_input_directory()
# files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
return {
"required": {
"image_base64": ("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
"text": ("STRING",{"multiline": True,"default": "","dynamicPrompts": True}),
}
}
RETURN_TYPES = ("IMAGE","STRING",)
RETURN_NAMES = ("image","text",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,False,)
OUTPUT_NODE = False
# 运行的函数
def run(self,image_base64,text):
image = base64_to_image(image_base64)
image=image.convert('RGB')
image=pil2tensor(image)
return (image,text,)
class EnhanceImage:
@@ -1203,42 +1024,6 @@ class EnhanceImage:
class LoadImages_:
@classmethod
def INPUT_TYPES(s):
return {"required":
{"images": ("IMAGEBASE64",),
},
}
CATEGORY = "♾️Mixlab/Image"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
RETURN_TYPES = ("IMAGE",)
FUNCTION = "load_image"
def load_image(self, images):
# print(images)
ims=[]
for im in images['base64']:
image = base64_to_image(im)
image=image.convert('RGB')
image=pil2tensor(image)
ims.append(image)
image1 = ims[0]
for image2 in ims[1:]:
if image1.shape[1:] != image2.shape[1:]:
image2 = comfy.utils.common_upscale(image2.movedim(-1, 1), image1.shape[2], image1.shape[1], "bilinear", "center").movedim(1, -1)
image1 = torch.cat((image1, image2), dim=0)
return (image1,)
'''
("STRING",{"multiline": False,"default": "Hello World!"})
对应 widgets.js 里:
@@ -1274,15 +1059,14 @@ class LoadImagesFromPath:
}
}
RETURN_TYPES = ('IMAGE','MASK','STRING','STRING',)
RETURN_NAMES = ("IMAGE","MASK","prompt_for_FloatingVideo","filepaths",)
RETURN_TYPES = ('IMAGE','MASK','STRING',)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Image"
# INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,True,False,True,)
OUTPUT_IS_LIST = (True,True,False,)
global watcher_folder
watcher_folder=None
@@ -1318,12 +1102,10 @@ class LoadImagesFromPath:
imgs=[]
masks=[]
file_names=[]
for im in sorted_files:
imgs.append(im['image'])
masks.append(im['mask'])
file_names.append(im['file_name'])
# print('index_variable',index_variable)
@@ -1331,12 +1113,11 @@ class LoadImagesFromPath:
if index_variable!=-1:
imgs=[imgs[index_variable]] if index_variable < len(imgs) else None
masks=[masks[index_variable]] if index_variable < len(masks) else None
file_names=[file_names[index_variable]] if index_variable < len(file_names) else None
except Exception as e:
print("发生了一个未知的错误:", str(e))
# print('#prompt::::',prompt)
return {"ui": {"seed": [1]}, "result":(imgs,masks,prompt,file_names,)}
return {"ui": {"seed": [1]}, "result":(imgs,masks,prompt,)}
# TODO 扩大选区的功能,重新输出mask
@@ -1588,7 +1369,7 @@ class Image3D:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/3D"
CATEGORY = "♾️Mixlab/Image"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,False,False,False,)
@@ -1698,37 +1479,6 @@ class FaceToMask:
return (mask,)
class CompositeImages:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"foreground": ("IMAGE",),
"mask":("MASK",),
"background": ("IMAGE",),
},
}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("IMAGE",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Layer"
# OUTPUT_IS_LIST = (True,)
def run(self, foreground,mask,background):
foreground= tensor2pil(foreground)
mask= tensor2pil(mask)
background= tensor2pil(background)
res=composite_images(foreground,background,mask)
return (pil2tensor(res),)
class EmptyLayer:
@classmethod
def INPUT_TYPES(s):
@@ -1824,7 +1574,7 @@ class NewLayer:
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"scale_option": (["width","height",'overall'],),
"image": (any_type,),
"image": ("IMAGE",),
},
"optional":{
"mask": ("MASK",{"default": None}),
@@ -1870,31 +1620,20 @@ class NewLayer:
return (layer_n,)
def createMask(image,x,y,w,h):
mask = Image.new("L", image.size)
pixels = mask.load()
# 遍历指定区域的像素,将其设置为黑色(0 表示黑色)
for i in range(int(x), int(x + w)):
for j in range(int(y), int(y + h)):
pixels[i, j] = 255
# mask.save("mask.png")
return mask
def splitImage(image, num):
width, height = image.size
num_rows = int(num ** 0.5)
num_cols = int(num / num_rows)
grid_width = int(width // num_cols)
grid_height = int(height // num_rows)
grid_width = width // num_cols
grid_height = height // num_rows
grid_coordinates = []
for i in range(num_rows):
for j in range(num_cols):
x = int(j * grid_width)
y = int(i * grid_height)
x = j * grid_width
y = i * grid_height
grid_coordinates.append((x, y, grid_width, grid_height))
return grid_coordinates
@@ -1949,8 +1688,8 @@ class SplitImage:
}
}
RETURN_TYPES = ("_GRID","_GRID","MASK",)
RETURN_NAMES = ("grids","grid","mask",)
RETURN_TYPES = ("_GRID","_GRID",)
RETURN_NAMES = ("grids","grid")
FUNCTION = "run"
@@ -1960,31 +1699,20 @@ class SplitImage:
# OUTPUT_IS_LIST = (True,)
def run(self,image,num,seed):
if type(seed) == list and len(seed)==1:
seed=seed[0]
image=tensor2pil(image)
grids=splitImage(image,num)
if seed>num:
num=seed % (num + 1)
if seed>=num:
num=int(seed / 500 * num)-1
else:
num=seed-1
print('#SplitImage',seed)
num=max(0,num)
num=min(num,len(grids)-1)
g=grids[num]
x,y,w,h=g
mask=createMask(image, x,y,w,h)
mask=pil2tensor(mask)
return (grids,g,mask,)
return (grids,g,)
@@ -2025,8 +1753,8 @@ class CenterImage:
}
}
RETURN_TYPES = ("_GRID","MASK",)
RETURN_NAMES = ("grid","mask",)
RETURN_TYPES = ("_GRID",)
RETURN_NAMES = ("grid",)
FUNCTION = "run"
@@ -2040,202 +1768,22 @@ class CenterImage:
grid=centerImage((left,top,right,bottom),canvas)
mask=createMask(canvas,left,top,canvas.width-left-right,canvas.height-top-bottom)
return (grid,pil2tensor(mask),)
class GridDisplayAndSave:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"labels": ("STRING",
{
"multiline": True,
"default": "",
"forceInput": True,
"dynamicPrompts": False
}),
"grids": ("_GRID",),
"image": ("IMAGE",),
"filename_prefix": ("STRING", {"default": "mixlab/grids"})
}
}
RETURN_TYPES = ( )
RETURN_NAMES = ( )
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Layer"
INPUT_IS_LIST = True
OUTPUT_NODE = True
# OUTPUT_IS_LIST = (True,)
def run(self,labels,grids,image,filename_prefix):
# print(image.shape)
img= tensor2pil(image[0])
for grid in grids:
draw_rectangle(img, grid, 'red',8)
#获取临时目录:temp
output_dir = folder_paths.get_temp_directory()
(
full_output_folder,
filename,
counter,
subfolder,
_,
) = folder_paths.get_save_image_path('tmp_', output_dir)
image_file = f"{filename}_{counter:05}.png"
image_path=os.path.join(full_output_folder, image_file)
# 保存图片
img.save(image_path,compress_level=6)
width, height = img.size
(
full_output_folder,
filename,
counter,
_,
_,
) = folder_paths.get_save_image_path(filename_prefix[0], output_dir)
return (grid,)
data_converted = [{
"label":labels[i],
"grid":[float(grids[i][0]),
float(grids[i][1]),
float(grids[i][2]),
float(grids[i][3])
]
} for i in range(len(grids))]
data={
"width":int(width),
"height":int(height),
"grids":data_converted
}
save_json_to_file(data,os.path.join(full_output_folder,f"${filename}_{counter:05}.json"))
return {"ui":{"image": [{
"filename": image_file,
"subfolder": subfolder,
"type":"temp"
}],
"json":[data["width"],data['height'],data["grids"]]
},"result": ()}
# return {"ui":{"image": [ ],
# },"result": ()}
class GridInput:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"grids": ("STRING",
{
"multiline": True,
"default": "",
"dynamicPrompts": False
}),
"padding":("INT",{
"default": 24,
"min": -500, #Minimum value
"max": 5000, #Maximum value
"step": 1, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
},
"optional":{
"width":("INT",{
"forceInput": True,
}),
"height":("INT",{
"forceInput": True,
}),
}
}
RETURN_TYPES = ("_GRID","STRING","IMAGE",)
RETURN_NAMES = ("grids","labels","image",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,True,False,)
OUTPUT_NODE = True
def run(self,grids,padding,width=[-1],height=[-1]):
# print(padding[0],grids[0])
width=width[0]
height=height[0]
grids=grids[0]
data=json.loads(grids)
grids=data['grids']
if width>-1:
data['width']=width
if height>-1:
data['height']=height
new_grids=[]
labels=[]
for g in grids:
labels.append(g['label'])
new_grids.append(padding_rectangle(g['grid'],padding[0]))
image = Image.new("RGB", (int(data['width']),int(data["height"])), "white")
im=pil2tensor(image)
# image=create_temp_file(im)
data_converted = [{
"label":labels[i],
"grid":[float(new_grids[i][0]),
float(new_grids[i][1]),
float(new_grids[i][2]),
float(new_grids[i][3])
]
} for i in range(len(new_grids))]
# 传递到前端节点的数据 报错,需要处理成 key:[x,x,x,x]
return {"ui":{
"json":[data["width"],data["height"],data_converted]
},"result": (new_grids,labels,im,)}
# return (new_grids,labels,pil2tensor(image),)
class GridOutput:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"grid": ("_GRID",),
},
"optional":{
"bg_image":("IMAGE",)
}
"grid": ("_GRID",)
}
}
RETURN_TYPES = ("INT","INT","INT","INT","MASK",)
RETURN_NAMES = ("x","y","width","height","mask",)
RETURN_TYPES = ("INT","INT","INT","INT",)
RETURN_NAMES = ("x","y","width","height",)
FUNCTION = "run"
@@ -2244,29 +1792,9 @@ class GridOutput:
INPUT_IS_LIST = False
# OUTPUT_IS_LIST = (True,)
def run(self,grid,bg_image=None):
def run(self,grid):
x,y,w,h=grid
x=int(x)
y=int(y)
w=int(w)
h=int(h)
masks=[]
if bg_image!=None:
for i in range(len(bg_image)):
im=bg_image[i]
#增加输出mask
im=tensor2pil(im)
mask=areaToMask(x,y,w,h,im)
mask=pil2tensor(mask)
masks.append(mask)
out=None
if len(masks)>0:
out = torch.cat(masks, dim=0)
return (x,y,w,h,out,)
return (x,y,w,h,)
class ShowLayer:
@classmethod
@@ -2358,14 +1886,10 @@ class MergeLayers:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"layers": ("LAYER",),
"images": ("IMAGE",),
},
"optional":{
"is_multiply_blend": ("BOOLEAN", {"default": False}),
}
"layers": ("LAYER",),
"images": ("IMAGE",),
},
}
RETURN_TYPES = ("IMAGE","MASK",)
@@ -2378,12 +1902,11 @@ class MergeLayers:
INPUT_IS_LIST = True
# OUTPUT_IS_LIST = (False,)
def run(self,layers,images,is_multiply_blend):
def run(self,layers,images):
bg_images=[]
masks=[]
is_multiply_blend=is_multiply_blend[0]
# print(len(images),images[0].shape)
# 1 torch.Size([2, 512, 512, 3])
# 4 torch.Size([1, 1024, 768, 3])
@@ -2414,8 +1937,6 @@ class MergeLayers:
layer_image=tensor2pil(image)
layer_mask=tensor2pil(mask)
# t=layer_image.convert("RGBA")
# t.save('test.png') 如果layerimage传入的是rgba,则是透明的
bg_image=merge_images(bg_image,
layer_image,
layer_mask,
@@ -2423,8 +1944,7 @@ class MergeLayers:
layer['y'],
layer['width'],
layer['height'],
layer['scale_option'],
is_multiply_blend
layer['scale_option']
)
final_mask=merge_images(final_mask,
@@ -2671,14 +2191,10 @@ class ResizeImage:
for ims in image:
for im in ims:
im=tensor2pil(im)
im=im.convert('RGB')
a_im,hex=get_average_color_image(im)
if average_color=='on':
fill_color=hex
im=resize_image(im,scale_option,w,h,fill_color)
im=im.convert('RGB')
a_im,hex=get_average_color_image(im)
im=pil2tensor(im)
imgs.append(im)
@@ -2743,93 +2259,21 @@ class GetImageSize_:
return {
"required": {
"image": ("IMAGE",),
},
"optional":{
"min_width":("INT", {
"default": 512,
"min":1, #Minimum value
"max": 2048, #Maximum value
"step": 8, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
})
},
}
}
RETURN_TYPES = ("INT", "INT","INT", "INT",)
RETURN_NAMES = ("width", "height","min_width", "min_height",)
RETURN_TYPES = ("INT", "INT")
RETURN_NAMES = ("width", "height")
FUNCTION = "get_size"
CATEGORY = "♾️Mixlab/Image"
def get_size(self, image,min_width):
def get_size(self, image):
_, height, width, _ = image.shape
# 如果比min_widht,还小,则输出 min width
if min_width>width:
im=tensor2pil(image)
im=resize_image(im,'width',min_width,min_width,"white")
im=im.convert('RGB')
return (width, height)
min_width,min_height=im.size
else:
min_width=width
min_height=height
return (width, height,min_width,min_height,)
class SaveImageAndMetadata:
def __init__(self):
self.output_dir = folder_paths.get_output_directory()
self.type = "output"
self.prefix_append = ""
self.compress_level = 4
@classmethod
def INPUT_TYPES(s):
return {"required":
{"images": ("IMAGE", ),
"filename_prefix": ("STRING", {"default": "Mixlab"}),
"metadata": (["disable","enable"],),
},
"hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
}
RETURN_TYPES = ()
FUNCTION = "save_images"
OUTPUT_NODE = True
CATEGORY = "♾️Mixlab/Output"
def save_images(self, images, filename_prefix="Mixlab",metadata="disable", prompt=None, extra_pnginfo=None):
filename_prefix += self.prefix_append
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0])
results = list()
for image in images:
i = 255. * image.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
metadata = None
if (not args.disable_metadata) and (metadata=="enable"):
print('##enable_metadata')
metadata = PngInfo()
if prompt is not None:
metadata.add_text("prompt", json.dumps(prompt))
if extra_pnginfo is not None:
for x in extra_pnginfo:
metadata.add_text(x, json.dumps(extra_pnginfo[x]))
file = f"{filename}_{counter:05}_.png"
img.save(os.path.join(full_output_folder, file), pnginfo=metadata, compress_level=self.compress_level)
results.append({
"filename": file,
"subfolder": subfolder,
"type": self.type
})
counter += 1
return { "ui": { "images": results } }
class ImageColorTransfer:
@classmethod
@@ -2837,7 +2281,6 @@ class ImageColorTransfer:
return {"required": {
"source": ("IMAGE",),
"target": ("IMAGE",),
"weight": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
},
}
@@ -2848,48 +2291,28 @@ class ImageColorTransfer:
FUNCTION = "run"
# 右键菜单目录
CATEGORY = "♾️Mixlab/Color"
CATEGORY = "♾️Mixlab/Image"
# 输入是否为列表
# INPUT_IS_LIST = True
INPUT_IS_LIST = True
# 输出是否为列表
# OUTPUT_IS_LIST = (True,)
OUTPUT_IS_LIST = (True,)
def run(self,source,target,weight):
def run(self,source,target):
res=[]
#batch-list
source_list = [source[i:i + 1, ...] for i in range(source.shape[0])]
target_list = [target[i:i + 1, ...] for i in range(target.shape[0])]
target=target[0][0]
print(target.shape)
target=tensor2pil(target)
# 长度纠正为相等
if len(target_list) != len(source_list):
target_list = target_list * (len(source_list) // len(target_list)) + target_list[:len(source_list) % len(target_list)]
for i in range(len(source_list)):
target=target_list[i]
source=source_list[i]
target=tensor2pil(target)
image=tensor2pil(source)
image_res=color_transfer(image,target)
# weight Blend image # contributors:@ning
blend_mask = Image.new(mode="L", size=image.size,
color=(round(weight * 255)))
blend_mask = ImageOps.invert(blend_mask)
img_result = Image.composite(image, image_res, blend_mask)
del image, image_res, blend_mask
img_result=pil2tensor(img_result)
res.append(img_result)
# list - batch
res=torch.cat(res, dim=0)
for ims in source:
for im in ims:
image=tensor2pil(im)
image=color_transfer(image,target)
image=pil2tensor(image)
res.append(image)
return (res,)
@@ -2916,26 +2339,15 @@ class SaveImageToLocal:
OUTPUT_NODE = True
CATEGORY = "♾️Mixlab/Output"
CATEGORY = "♾️Mixlab/Image"
def save_images(self, images,file_path , prompt=None, extra_pnginfo=None):
filename_prefix = os.path.basename(file_path)
if file_path=='':
filename_prefix="ComfyUI"
filename_prefix, _ = os.path.splitext(filename_prefix)
_, extension = os.path.splitext(file_path)
if extension:
# 是文件名,需要处理
file_path=os.path.dirname(file_path)
# filename_prefix=
full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0])
if not os.path.exists(file_path):
# 使用os.makedirs函数创建新目录
os.makedirs(file_path)
@@ -2981,7 +2393,6 @@ class SaveImageToLocal:
})
else:
fp=os.path.join(file_path, file)
if os.path.exists(fp):
file = f"{filename}_{counter:05}_{generate_random_string(8)}.png"
+3
View File
@@ -42,6 +42,9 @@ else:
_available=True
lama_path=os.path.join(folder_paths.models_dir, "lama")
if not os.path.exists(lama_path):
os.mkdir(lama_path)
llma_model_path=os.path.join(folder_paths.models_dir, "lama/big-lama.pt")
if not os.path.exists(llma_model_path):
+5 -108
View File
@@ -2,14 +2,16 @@
import scipy.ndimage
import torch
from nodes import MAX_RESOLUTION
import numpy as np
# from PIL import Image, ImageDraw
from PIL import Image, ImageOps
from comfy.cli_args import args
import cv2,os
from nodes import MAX_RESOLUTION, SaveImage, common_ksampler
import folder_paths,random
import cv2
# Tensor to PIL
def tensor2pil(image):
@@ -20,19 +22,6 @@ def pil2tensor(image):
return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
def add_masks(mask1, mask2):
mask1 = mask1.cpu()
mask2 = mask2.cpu()
cv2_mask1 = np.array(mask1) * 255
cv2_mask2 = np.array(mask2) * 255
if cv2_mask1.shape == cv2_mask2.shape:
cv2_mask = cv2.add(cv2_mask1, cv2_mask2)
return torch.clamp(torch.from_numpy(cv2_mask) / 255.0, min=0, max=1)
else:
return mask1
def grow(mask, expand, tapered_corners):
c = 0 if tapered_corners else 1
kernel = np.array([[c, 1, c],
@@ -69,35 +58,6 @@ def combine(destination, source, x, y):
return output
class PreviewMask_(SaveImage):
def __init__(self):
self.output_dir = folder_paths.get_temp_directory()
self.type = "temp"
self.prefix_append =''.join(random.choice("abcdehijklmnopqrstupvxyzfg") for x in range(5))
self.compress_level = 4
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"mask": ("MASK",),
}
}
RETURN_TYPES = ()
OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Mask"
# 运行的函数
def run(self, mask ):
img=tensor2pil(mask)
img=img.convert('RGB')
img=pil2tensor(img)
return self.save_images(img, 'temp_', None, None)
class OutlineMask:
@classmethod
@@ -127,69 +87,6 @@ class OutlineMask:
return (m3,)
class MaskListReplace:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"masks": ("MASK",),
"mask_replace": ("MASK",),
"start_index":("INT", {"default": 0, "min": 0, "step": 1}),
"end_index":("INT", {"default": 0, "min": 0, "step": 1}),
"invert": ("BOOLEAN", {"default": False}),
}
}
RETURN_TYPES = ("MASK",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Video"
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,)
def run(self, masks,mask_replace,start_index,end_index,invert):
mask_replace=mask_replace[0]
start_index=start_index[0]
end_index=end_index[0]
invert=invert[0]
new_masks=[]
for i in range(len(masks)):
if i>=start_index and i<=end_index:
if invert:
new_masks.append(masks[i])
else:
new_masks.append(mask_replace)
else:
if invert:
new_masks.append(mask_replace)
else:
new_masks.append(masks[i])
return (new_masks,)
class MaskListMerge:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"masks": ("MASK",),
}
}
RETURN_TYPES = ("MASK",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Mask"
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (False,)
def run(self, masks):
mask=masks[0]
if isinstance(masks, list):
for m in masks:
# print(m.shape)
mask = add_masks(mask, m)
return (mask,)
class FeatheredMask:
+5 -241
View File
@@ -6,12 +6,6 @@ from urllib import request, parse
import folder_paths
from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence, ImageFont
from PIL.PngImagePlugin import PngInfo
import hashlib
import requests
import json
# def queue_prompt(prompt_workflow):
# p = {"prompt": prompt_workflow}
# data = json.dumps(p).encode('utf-8')
@@ -21,7 +15,6 @@ import json
embeddings_path=os.path.join(folder_paths.models_dir, "embeddings")
def get_files_with_extension(directory, extension):
file_list = []
for root, dirs, files in os.walk(directory):
for file in files:
@@ -35,50 +28,6 @@ def join_with_(text_list,delimiter):
return joined_text
def load_json(file_path):
try:
with open(file_path, 'r') as json_file:
data = json.load(json_file)
return data
except FileNotFoundError:
print(f"File not found: {file_path}")
return None
except json.JSONDecodeError:
print(f"Error decoding JSON in file: {file_path}")
return None
def save_json(data_dict, file_path):
try:
with open(file_path, 'w') as json_file:
json.dump(data_dict, json_file, indent=4)
print(f"Data saved to {file_path}")
except Exception as e:
print(f"Error saving JSON to file: {e}")
# pysss的lora加载器
# def get_model_version_info(hash_value):
# # http://127.0.0.1:1082
# proxies = {'http': 'http://127.0.0.1:1082', 'https': 'https://127.0.0.1:1082'}
# api_url = f"https://civitai.com/api/v1/model-versions/by-hash/{hash_value}"
# print(api_url)
# response = requests.get(api_url,proxies=proxies, verify=False)
# if response.status_code == 200:
# return response.json()
# else:
# return None
# def calculate_sha256(file_path):
# sha256_hash = hashlib.sha256()
# with open(file_path, "rb") as f:
# for chunk in iter(lambda: f.read(4096), b""):
# sha256_hash.update(chunk)
# return sha256_hash.hexdigest()
class AnyType(str):
"""A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
@@ -189,7 +138,7 @@ class PromptImage:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Output"
CATEGORY = "♾️Mixlab/Prompt"
# 运行的函数
def run(self,prompts,images,save_to_image):
@@ -436,92 +385,16 @@ class RandomPrompt:
return {"ui": {"prompts": prompts}, "result": (prompts,)}
# class LoraPrompt:
# @classmethod
# def INPUT_TYPES(s):
# return {
# "required": {
# "lora_name":(sorted(folder_paths.get_filename_list("loras"), key=str.lower),),
# "weight": ("FLOAT", {"default": 1, "min": -2, "max": 2,"step":0.01 ,"display": "slider"}),
# "force_update": ("BOOLEAN", {"default": False}),
# },
# }
# RETURN_TYPES = ("STRING","STRING",any_type)
# RETURN_NAMES = ("lora_name","prompt","tags",)
# FUNCTION = "run"
# CATEGORY = "♾️Mixlab/Prompt"
# OUTPUT_IS_LIST = (False,False,True,)
# # OUTPUT_NODE = True
# # 运行的函数
# def run(self,lora_name,weight,force_update=False):
# # print('##LoraPrompt',__file__)
# # 从本地数据库读取
# json_tags_path = os.path.join(os.path.dirname(os.path.dirname(__file__)),r'data/loras_tags.json')
# if not os.path.exists(json_tags_path):
# save_json({},json_tags_path)
# lora_tags = load_json(json_tags_path)
# output_tags = lora_tags.get(lora_name, None) if lora_tags is not None else None
# if output_tags is not None:
# output_tags = ",".join(output_tags)
# print("trainedWords:",output_tags)
# else:
# output_tags = ""
# lora_path = folder_paths.get_full_path("loras", lora_name)
# if output_tags == "" or force_update:
# print("calculating lora hash")
# LORAsha256 = calculate_sha256(lora_path)
# print("requesting infos")
# model_info = get_model_version_info(LORAsha256)
# if model_info is not None:
# if "trainedWords" in model_info:
# print("tags found!")
# if lora_tags is None:
# lora_tags = {}
# lora_tags[lora_name] = model_info["trainedWords"]
# save_json(lora_tags,json_tags_path)
# output_tags = ",".join(model_info["trainedWords"])
# print("trainedWords:",output_tags)
# else:
# print("No informations found.")
# if lora_tags is None:
# lora_tags = {}
# lora_tags[lora_name] = []
# save_json(lora_tags,json_tags_path)
# weight = round(weight, 3)
# prompt=[]
# for p in output_tags.split(','):
# if weight!=1:
# prompt.append('('+p+':'+str(weight)+')')
# else:
# prompt.append(p)
# prompt=",".join(prompt)
# return (lora_name,prompt,output_tags.split(','),)
class EmbeddingPrompt:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"embedding":(folder_paths.get_filename_list("embeddings"),),
"embedding":(get_files_with_extension(embeddings_path,'.pt'),),
"weight": ("FLOAT", {"default": 1, "min": -2, "max": 2,"step":0.01 ,"display": "slider"}),
},
@@ -546,119 +419,14 @@ class EmbeddingPrompt:
# return (new_prompt)
return (prompt,)
# RETURN_TYPES = (any_type,)
# conditioning :提示,正向or负向
# clip:clip模型
# gligen_textbox_model:gligen模型
# grids:矩形框的集合
# labels:每个矩形框对应的标签的集合
# index:选取第几个矩形框作为gligen的box
class GLIGENTextBoxApply_Advanced:
@classmethod
def INPUT_TYPES(s):
return {"required": {"conditioning": ("CONDITIONING", ),
"clip": ("CLIP", ),
"gligen_textbox_model": ("GLIGEN", ),
"grids": ("_GRID",),
"labels": ("STRING",
{
"multiline": True,
"default": "",
"forceInput": True
}),
"index": ("INT", {"default": -1, "min": -1, "max": 300, "step": 1}),
"max_size": ("INT", {"default": 8, "min": 1, "max": 300, "step": 1}),
"random_shuffle":(["on","off"],),
},
"optional":{
"seed": (any_type, {"default": 0, "min": 0, "max": 0xffffffffffffffff,"step": 1}),
}
}
RETURN_TYPES = ("CONDITIONING","STRING",)
RETURN_NAMES = ("CONDITIONING","label",)
FUNCTION = "run"
# INPUT_IS_LIST = True
CATEGORY = "♾️Mixlab/Prompt"
def run(self, conditioning, clip, gligen_textbox_model, grids, labels, index,max_size,random_shuffle,seed=0):
# print('grids',grids)
# conditioning=conditioning[0]
# clip=clip[0]
# gligen_textbox_model=gligen_textbox_model[0]
# index=index[0]
# max_size=max_size[0]
# random_shuffle=random_shuffle[0]
texts=labels
if index>-1:
texts=[labels[index]]
grids=[grids[index]]
if random_shuffle=='on':
sss=[[texts[i],grids[i]] for i in range(len(texts))]
random.shuffle(sss)
texts=[s[0] for s in sss]
grids=[s[1] for s in sss]
if len(texts) > max_size:
texts = texts[:max_size]
c = []
for t in conditioning:
n = [t[0], t[1].copy()]
# 多个
position_params=[]
for i in range(len(texts)):
text=texts[i]
grid=grids[i]
x,y,width,height=grid
# print(text)
cond, cond_pooled = clip.encode_from_tokens(clip.tokenize(text), return_pooled=True)
position_params =position_params+ [(cond_pooled, height // 8, width // 8, y // 8, x // 8)]
# 前一个
prev = []
if "gligen" in n[1]:
prev = n[1]['gligen'][2]
n[1]['gligen'] = ("position", gligen_textbox_model, prev + position_params)
# print('gligen',n)
c.append(n)
# 下面这个写法有bug
# for i in range(len(texts)):
# text=texts[i]
# grid=grids[i]
# x,y,width,height=grid
# cond, cond_pooled = clip.encode_from_tokens(clip.tokenize(text), return_pooled=True)
# for t in conditioning:
# n = [t[0], t[1].copy()]
# position_params = [(cond_pooled, height // 8, width // 8, y // 8, x // 8)]
# prev = []
# if "gligen" in n[1]:
# prev = n[1]['gligen'][2]
# n[1]['gligen'] = ("position", gligen_textbox_model, prev + position_params)
# c.append(n)
return (c,texts, )
RETURN_TYPES = (any_type,)
class JoinWithDelimiter:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"text_list": (any_type,),
"delimiter":(["newline","comma","backslash","space"],),
"delimiter":(["newline","comma"],),
},
}
@@ -666,7 +434,7 @@ class JoinWithDelimiter:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Text"
CATEGORY = "♾️Mixlab/Prompt"
INPUT_IS_LIST = True # 当true的时候,输入时list,当false的时候,如果输入是list,则会自动包一层for循环调用
OUTPUT_IS_LIST = (False,)
@@ -677,10 +445,6 @@ class JoinWithDelimiter:
delimiter='\n'
elif delimiter=='comma':
delimiter=','
elif delimiter=='backslash':
delimiter='\\'
elif delimiter=='space':
delimiter=' '
t=''
if isinstance(text_list, list):
t=join_with_(text_list,delimiter)
+3 -533
View File
@@ -8,467 +8,6 @@ import comfy.utils
import numpy as np
import torch
from huggingface_hub import hf_hub_download
import torch.nn as nn
import torch.nn.functional as F
from torchvision.transforms.functional import normalize
# BRIA-RMBG-1.4 / briarmbg.py
class REBNCONV(nn.Module):
def __init__(self,in_ch=3,out_ch=3,dirate=1,stride=1):
super(REBNCONV,self).__init__()
self.conv_s1 = nn.Conv2d(in_ch,out_ch,3,padding=1*dirate,dilation=1*dirate,stride=stride)
self.bn_s1 = nn.BatchNorm2d(out_ch)
self.relu_s1 = nn.ReLU(inplace=True)
def forward(self,x):
hx = x
xout = self.relu_s1(self.bn_s1(self.conv_s1(hx)))
return xout
## upsample tensor 'src' to have the same spatial size with tensor 'tar'
def _upsample_like(src,tar):
src = F.interpolate(src,size=tar.shape[2:],mode='bilinear')
return src
### RSU-7 ###
class RSU7(nn.Module):
def __init__(self, in_ch=3, mid_ch=12, out_ch=3, img_size=512):
super(RSU7,self).__init__()
self.in_ch = in_ch
self.mid_ch = mid_ch
self.out_ch = out_ch
self.rebnconvin = REBNCONV(in_ch,out_ch,dirate=1) ## 1 -> 1/2
self.rebnconv1 = REBNCONV(out_ch,mid_ch,dirate=1)
self.pool1 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv2 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool2 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv3 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool3 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv4 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool4 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv5 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool5 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv6 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.rebnconv7 = REBNCONV(mid_ch,mid_ch,dirate=2)
self.rebnconv6d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv5d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv4d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv3d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv2d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv1d = REBNCONV(mid_ch*2,out_ch,dirate=1)
def forward(self,x):
b, c, h, w = x.shape
hx = x
hxin = self.rebnconvin(hx)
hx1 = self.rebnconv1(hxin)
hx = self.pool1(hx1)
hx2 = self.rebnconv2(hx)
hx = self.pool2(hx2)
hx3 = self.rebnconv3(hx)
hx = self.pool3(hx3)
hx4 = self.rebnconv4(hx)
hx = self.pool4(hx4)
hx5 = self.rebnconv5(hx)
hx = self.pool5(hx5)
hx6 = self.rebnconv6(hx)
hx7 = self.rebnconv7(hx6)
hx6d = self.rebnconv6d(torch.cat((hx7,hx6),1))
hx6dup = _upsample_like(hx6d,hx5)
hx5d = self.rebnconv5d(torch.cat((hx6dup,hx5),1))
hx5dup = _upsample_like(hx5d,hx4)
hx4d = self.rebnconv4d(torch.cat((hx5dup,hx4),1))
hx4dup = _upsample_like(hx4d,hx3)
hx3d = self.rebnconv3d(torch.cat((hx4dup,hx3),1))
hx3dup = _upsample_like(hx3d,hx2)
hx2d = self.rebnconv2d(torch.cat((hx3dup,hx2),1))
hx2dup = _upsample_like(hx2d,hx1)
hx1d = self.rebnconv1d(torch.cat((hx2dup,hx1),1))
return hx1d + hxin
### RSU-6 ###
class RSU6(nn.Module):
def __init__(self, in_ch=3, mid_ch=12, out_ch=3):
super(RSU6,self).__init__()
self.rebnconvin = REBNCONV(in_ch,out_ch,dirate=1)
self.rebnconv1 = REBNCONV(out_ch,mid_ch,dirate=1)
self.pool1 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv2 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool2 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv3 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool3 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv4 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool4 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv5 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.rebnconv6 = REBNCONV(mid_ch,mid_ch,dirate=2)
self.rebnconv5d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv4d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv3d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv2d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv1d = REBNCONV(mid_ch*2,out_ch,dirate=1)
def forward(self,x):
hx = x
hxin = self.rebnconvin(hx)
hx1 = self.rebnconv1(hxin)
hx = self.pool1(hx1)
hx2 = self.rebnconv2(hx)
hx = self.pool2(hx2)
hx3 = self.rebnconv3(hx)
hx = self.pool3(hx3)
hx4 = self.rebnconv4(hx)
hx = self.pool4(hx4)
hx5 = self.rebnconv5(hx)
hx6 = self.rebnconv6(hx5)
hx5d = self.rebnconv5d(torch.cat((hx6,hx5),1))
hx5dup = _upsample_like(hx5d,hx4)
hx4d = self.rebnconv4d(torch.cat((hx5dup,hx4),1))
hx4dup = _upsample_like(hx4d,hx3)
hx3d = self.rebnconv3d(torch.cat((hx4dup,hx3),1))
hx3dup = _upsample_like(hx3d,hx2)
hx2d = self.rebnconv2d(torch.cat((hx3dup,hx2),1))
hx2dup = _upsample_like(hx2d,hx1)
hx1d = self.rebnconv1d(torch.cat((hx2dup,hx1),1))
return hx1d + hxin
### RSU-5 ###
class RSU5(nn.Module):
def __init__(self, in_ch=3, mid_ch=12, out_ch=3):
super(RSU5,self).__init__()
self.rebnconvin = REBNCONV(in_ch,out_ch,dirate=1)
self.rebnconv1 = REBNCONV(out_ch,mid_ch,dirate=1)
self.pool1 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv2 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool2 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv3 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool3 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv4 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.rebnconv5 = REBNCONV(mid_ch,mid_ch,dirate=2)
self.rebnconv4d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv3d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv2d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv1d = REBNCONV(mid_ch*2,out_ch,dirate=1)
def forward(self,x):
hx = x
hxin = self.rebnconvin(hx)
hx1 = self.rebnconv1(hxin)
hx = self.pool1(hx1)
hx2 = self.rebnconv2(hx)
hx = self.pool2(hx2)
hx3 = self.rebnconv3(hx)
hx = self.pool3(hx3)
hx4 = self.rebnconv4(hx)
hx5 = self.rebnconv5(hx4)
hx4d = self.rebnconv4d(torch.cat((hx5,hx4),1))
hx4dup = _upsample_like(hx4d,hx3)
hx3d = self.rebnconv3d(torch.cat((hx4dup,hx3),1))
hx3dup = _upsample_like(hx3d,hx2)
hx2d = self.rebnconv2d(torch.cat((hx3dup,hx2),1))
hx2dup = _upsample_like(hx2d,hx1)
hx1d = self.rebnconv1d(torch.cat((hx2dup,hx1),1))
return hx1d + hxin
### RSU-4 ###
class RSU4(nn.Module):
def __init__(self, in_ch=3, mid_ch=12, out_ch=3):
super(RSU4,self).__init__()
self.rebnconvin = REBNCONV(in_ch,out_ch,dirate=1)
self.rebnconv1 = REBNCONV(out_ch,mid_ch,dirate=1)
self.pool1 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv2 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.pool2 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.rebnconv3 = REBNCONV(mid_ch,mid_ch,dirate=1)
self.rebnconv4 = REBNCONV(mid_ch,mid_ch,dirate=2)
self.rebnconv3d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv2d = REBNCONV(mid_ch*2,mid_ch,dirate=1)
self.rebnconv1d = REBNCONV(mid_ch*2,out_ch,dirate=1)
def forward(self,x):
hx = x
hxin = self.rebnconvin(hx)
hx1 = self.rebnconv1(hxin)
hx = self.pool1(hx1)
hx2 = self.rebnconv2(hx)
hx = self.pool2(hx2)
hx3 = self.rebnconv3(hx)
hx4 = self.rebnconv4(hx3)
hx3d = self.rebnconv3d(torch.cat((hx4,hx3),1))
hx3dup = _upsample_like(hx3d,hx2)
hx2d = self.rebnconv2d(torch.cat((hx3dup,hx2),1))
hx2dup = _upsample_like(hx2d,hx1)
hx1d = self.rebnconv1d(torch.cat((hx2dup,hx1),1))
return hx1d + hxin
### RSU-4F ###
class RSU4F(nn.Module):
def __init__(self, in_ch=3, mid_ch=12, out_ch=3):
super(RSU4F,self).__init__()
self.rebnconvin = REBNCONV(in_ch,out_ch,dirate=1)
self.rebnconv1 = REBNCONV(out_ch,mid_ch,dirate=1)
self.rebnconv2 = REBNCONV(mid_ch,mid_ch,dirate=2)
self.rebnconv3 = REBNCONV(mid_ch,mid_ch,dirate=4)
self.rebnconv4 = REBNCONV(mid_ch,mid_ch,dirate=8)
self.rebnconv3d = REBNCONV(mid_ch*2,mid_ch,dirate=4)
self.rebnconv2d = REBNCONV(mid_ch*2,mid_ch,dirate=2)
self.rebnconv1d = REBNCONV(mid_ch*2,out_ch,dirate=1)
def forward(self,x):
hx = x
hxin = self.rebnconvin(hx)
hx1 = self.rebnconv1(hxin)
hx2 = self.rebnconv2(hx1)
hx3 = self.rebnconv3(hx2)
hx4 = self.rebnconv4(hx3)
hx3d = self.rebnconv3d(torch.cat((hx4,hx3),1))
hx2d = self.rebnconv2d(torch.cat((hx3d,hx2),1))
hx1d = self.rebnconv1d(torch.cat((hx2d,hx1),1))
return hx1d + hxin
class myrebnconv(nn.Module):
def __init__(self, in_ch=3,
out_ch=1,
kernel_size=3,
stride=1,
padding=1,
dilation=1,
groups=1):
super(myrebnconv,self).__init__()
self.conv = nn.Conv2d(in_ch,
out_ch,
kernel_size=kernel_size,
stride=stride,
padding=padding,
dilation=dilation,
groups=groups)
self.bn = nn.BatchNorm2d(out_ch)
self.rl = nn.ReLU(inplace=True)
def forward(self,x):
return self.rl(self.bn(self.conv(x)))
class BriaRMBG(nn.Module):
def __init__(self,in_ch=3,out_ch=1):
super(BriaRMBG,self).__init__()
self.conv_in = nn.Conv2d(in_ch,64,3,stride=2,padding=1)
self.pool_in = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage1 = RSU7(64,32,64)
self.pool12 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage2 = RSU6(64,32,128)
self.pool23 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage3 = RSU5(128,64,256)
self.pool34 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage4 = RSU4(256,128,512)
self.pool45 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage5 = RSU4F(512,256,512)
self.pool56 = nn.MaxPool2d(2,stride=2,ceil_mode=True)
self.stage6 = RSU4F(512,256,512)
# decoder
self.stage5d = RSU4F(1024,256,512)
self.stage4d = RSU4(1024,128,256)
self.stage3d = RSU5(512,64,128)
self.stage2d = RSU6(256,32,64)
self.stage1d = RSU7(128,16,64)
self.side1 = nn.Conv2d(64,out_ch,3,padding=1)
self.side2 = nn.Conv2d(64,out_ch,3,padding=1)
self.side3 = nn.Conv2d(128,out_ch,3,padding=1)
self.side4 = nn.Conv2d(256,out_ch,3,padding=1)
self.side5 = nn.Conv2d(512,out_ch,3,padding=1)
self.side6 = nn.Conv2d(512,out_ch,3,padding=1)
# self.outconv = nn.Conv2d(6*out_ch,out_ch,1)
def forward(self,x):
hx = x
hxin = self.conv_in(hx)
#hx = self.pool_in(hxin)
#stage 1
hx1 = self.stage1(hxin)
hx = self.pool12(hx1)
#stage 2
hx2 = self.stage2(hx)
hx = self.pool23(hx2)
#stage 3
hx3 = self.stage3(hx)
hx = self.pool34(hx3)
#stage 4
hx4 = self.stage4(hx)
hx = self.pool45(hx4)
#stage 5
hx5 = self.stage5(hx)
hx = self.pool56(hx5)
#stage 6
hx6 = self.stage6(hx)
hx6up = _upsample_like(hx6,hx5)
#-------------------- decoder --------------------
hx5d = self.stage5d(torch.cat((hx6up,hx5),1))
hx5dup = _upsample_like(hx5d,hx4)
hx4d = self.stage4d(torch.cat((hx5dup,hx4),1))
hx4dup = _upsample_like(hx4d,hx3)
hx3d = self.stage3d(torch.cat((hx4dup,hx3),1))
hx3dup = _upsample_like(hx3d,hx2)
hx2d = self.stage2d(torch.cat((hx3dup,hx2),1))
hx2dup = _upsample_like(hx2d,hx1)
hx1d = self.stage1d(torch.cat((hx2dup,hx1),1))
#side output
d1 = self.side1(hx1d)
d1 = _upsample_like(d1,x)
d2 = self.side2(hx2d)
d2 = _upsample_like(d2,x)
d3 = self.side3(hx3d)
d3 = _upsample_like(d3,x)
d4 = self.side4(hx4d)
d4 = _upsample_like(d4,x)
d5 = self.side5(hx5d)
d5 = _upsample_like(d5,x)
d6 = self.side6(hx6)
d6 = _upsample_like(d6,x)
return [F.sigmoid(d1), F.sigmoid(d2), F.sigmoid(d3), F.sigmoid(d4), F.sigmoid(d5), F.sigmoid(d6)],[hx1d,hx2d,hx3d,hx4d,hx5d,hx6]
U2NET_HOME=os.path.join(folder_paths.models_dir, "rembg")
os.environ["U2NET_HOME"] = U2NET_HOME
@@ -509,70 +48,6 @@ except:
_available=False
def briarmbg_run(images=[]):
mroot=os.path.join(folder_paths.models_dir, "rembg")
m=os.path.join(mroot,'briarmbg.pth')
if os.path.exists(m)==False:
# 下载
m1=hf_hub_download("briaai/RMBG-1.4",
local_dir=mroot,
filename='model.pth',
local_dir_use_symlinks=False,
endpoint='https://hf-mirror.com')
os.rename(m1, m)
net=BriaRMBG()
if torch.cuda.is_available():
net.load_state_dict(torch.load(m))
net=net.cuda()
else:
net.load_state_dict(torch.load(m,map_location="cpu"))
net.eval()
masks=[]
rgba_images=[]
rgb_images=[]
for orig_image in images:
w,h = orig_im_size = orig_image.size
image = orig_image.convert('RGB')
model_input_size = (1024, 1024)
image = image.resize(model_input_size, Image.BILINEAR)
im_np = np.array(image)
im_tensor = torch.tensor(im_np, dtype=torch.float32).permute(2,0,1)
im_tensor = torch.unsqueeze(im_tensor,0)
im_tensor = torch.divide(im_tensor,255.0)
im_tensor = normalize(im_tensor,[0.5,0.5,0.5],[1.0,1.0,1.0])
if torch.cuda.is_available():
im_tensor=im_tensor.cuda()
result=net(im_tensor)
result = torch.squeeze(F.interpolate(result[0][0], size=(h,w), mode='bilinear') ,0)
ma = torch.max(result)
mi = torch.min(result)
result = (result-mi)/(ma-mi)
im_array = (result*255).cpu().data.numpy().astype(np.uint8)
mask = Image.fromarray(np.squeeze(im_array))
# mask.save('test.png')
# mask=tensor2pil(result)
mask=mask.convert('L')
masks.append(mask)
# rgba图
image_rgba =orig_image.convert("RGBA")
image_rgba.putalpha(mask)
rgba_images.append(image_rgba)
#rgb
rgb_image = Image.new("RGB", image_rgba.size, (0, 0, 0))
rgb_image.paste(image_rgba, mask=image_rgba.split()[3])
rgb_images.append(rgb_image)
return (masks,rgba_images,rgb_images)
def run_bg(model_name= "unet",images=[]):
# model_name = "unet" # "isnet-general-use"
rembg_session = new_session(model_name)
@@ -643,16 +118,14 @@ class RembgNode_:
def INPUT_TYPES(s):
return {"required": {
"image": ("IMAGE",),
"model_name": ([
"briarmbg",
"u2net",
"model_name": (["u2net",
"u2netp",
"u2net_human_seg",
"u2net_cloth_seg",
"silueta",
"isnet-general-use",
"isnet-anime",
# "sam"
],),
},
@@ -680,10 +153,7 @@ class RembgNode_:
im=tensor2pil(im)
images.append(im)
if model_name=='briarmbg':
masks,rgba_images,rgb_images=briarmbg_run(images)
else:
masks,rgba_images,rgb_images=run_bg(model_name,images)
masks,rgba_images,rgb_images=run_bg(model_name,images)
masks=[pil2tensor(m) for m in masks]
+2 -2
View File
@@ -93,7 +93,7 @@ class ScreenShareNode:
RETURN_NAMES = ("IMAGE","PROMPT","FLOAT","INT")
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Screen"
CATEGORY = "♾️Mixlab/Image"
# INPUT_IS_LIST = True
OUTPUT_IS_LIST = (False,False,False,False)
@@ -118,7 +118,7 @@ class FloatingVideo:
OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Screen"
CATEGORY = "♾️Mixlab/Image"
# INPUT_IS_LIST = True
# OUTPUT_IS_LIST = (False,False,)
-503
View File
@@ -1,503 +0,0 @@
import comfy
import torch
from dataclasses import dataclass
import torch.nn as nn
from comfy.model_patcher import ModelPatcher
import comfy.ops
from typing import Union
import comfy.sample
import latent_preview
import comfy.utils
T = torch.Tensor
from .VisualStylePrompting.attention_functions import VisualStyleProcessor
class ApplyVisualStylePrompting:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"reference_image": ("IMAGE",),
"reference_image_text": ("STRING", {"multiline": True}),
"model": ("MODEL",),
"clip": ("CLIP", ),
"vae": ("VAE", ),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING", ),
"enabled": ("BOOLEAN", {"default": True}),
"denoise": ("FLOAT", {"default": 1., "min": 0., "max": 1., "step": 1e-2}),
"batch_size": ("INT", {"default": 1, "min": 1, "max": 4096,"step":2})
}
}
RETURN_TYPES = ("MODEL", "CONDITIONING","CONDITIONING", "LATENT")
RETURN_NAMES = ("model", "positive", "negative", "latents")
CATEGORY = "♾️Mixlab/Style"
FUNCTION = "run"
def run(
self,
reference_image,
reference_image_text,
model: comfy.model_patcher.ModelPatcher,
clip,
vae,
positive,
negative,
enabled,
denoise,
batch_size=1
):
tokens = clip.tokenize(reference_image_text)
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
reference_image_prompt=[[cond, {"pooled_output": pooled}]]
reference_image = reference_image.repeat(((batch_size+1)//2, 1,1,1))
self.model = model
reference_latent = vae.encode(reference_image[:,:,:,:3])
for n, m in model.model.diffusion_model.named_modules():
if m.__class__.__name__ == "CrossAttention":
processor = VisualStyleProcessor(m, enabled=enabled)
setattr(m, 'forward', processor.visual_style_forward)
conditioning_prompt = reference_image_prompt + positive
negative_prompt = negative * 2
latents = torch.zeros_like(reference_latent)
latents = torch.cat([latents] * 2)
if denoise < 1.0:
latents[::1] = reference_latent[:1]
else:
latents[::2] = reference_latent
denoise_mask = torch.ones_like(latents)[:, :1, ...] * denoise
denoise_mask[0] = 0.
return (model, conditioning_prompt, negative_prompt, {"samples": latents, "noise_mask": denoise_mask})
def exists(val):
return val is not None
def default(val, d):
if exists(val):
return val
return d
class StyleAlignedArgs:
def __init__(self, share_attn: str) -> None:
self.adain_keys = "k" in share_attn
self.adain_values = "v" in share_attn
self.adain_queries = "q" in share_attn
share_attention: bool = True
adain_queries: bool = True
adain_keys: bool = True
adain_values: bool = True
def expand_first(
feat: T,
scale=1.0,
) -> T:
"""
Expand the first element so it has the same shape as the rest of the batch.
"""
b = feat.shape[0]
feat_style = torch.stack((feat[0], feat[b // 2])).unsqueeze(1)
if scale == 1:
feat_style = feat_style.expand(2, b // 2, *feat.shape[1:])
else:
feat_style = feat_style.repeat(1, b // 2, 1, 1, 1)
feat_style = torch.cat([feat_style[:, :1], scale * feat_style[:, 1:]], dim=1)
return feat_style.reshape(*feat.shape)
def concat_first(feat: T, dim=2, scale=1.0) -> T:
"""
concat the the feature and the style feature expanded above
"""
feat_style = expand_first(feat, scale=scale)
return torch.cat((feat, feat_style), dim=dim)
def calc_mean_std(feat, eps: float = 1e-5) -> "tuple[T, T]":
feat_std = (feat.var(dim=-2, keepdims=True) + eps).sqrt()
feat_mean = feat.mean(dim=-2, keepdims=True)
return feat_mean, feat_std
def adain(feat: T) -> T:
feat_mean, feat_std = calc_mean_std(feat)
feat_style_mean = expand_first(feat_mean)
feat_style_std = expand_first(feat_std)
feat = (feat - feat_mean) / feat_std
feat = feat * feat_style_std + feat_style_mean
return feat
class SharedAttentionProcessor:
def __init__(self, args: StyleAlignedArgs, scale: float):
self.args = args
self.scale = scale
def __call__(self, q, k, v, extra_options):
if self.args.adain_queries:
q = adain(q)
if self.args.adain_keys:
k = adain(k)
if self.args.adain_values:
v = adain(v)
if self.args.share_attention:
k = concat_first(k, -2, scale=self.scale)
v = concat_first(v, -2)
return q, k, v
def get_norm_layers(
layer: nn.Module,
norm_layers_: "dict[str, list[Union[nn.GroupNorm, nn.LayerNorm]]]",
share_layer_norm: bool,
share_group_norm: bool,
):
if isinstance(layer, nn.LayerNorm) and share_layer_norm:
norm_layers_["layer"].append(layer)
if isinstance(layer, nn.GroupNorm) and share_group_norm:
norm_layers_["group"].append(layer)
else:
for child_layer in layer.children():
get_norm_layers(
child_layer, norm_layers_, share_layer_norm, share_group_norm
)
def register_norm_forward(
norm_layer: Union[nn.GroupNorm, nn.LayerNorm],
) -> Union[nn.GroupNorm, nn.LayerNorm]:
if not hasattr(norm_layer, "orig_forward"):
setattr(norm_layer, "orig_forward", norm_layer.forward)
orig_forward = norm_layer.orig_forward
def forward_(hidden_states: T) -> T:
n = hidden_states.shape[-2]
hidden_states = concat_first(hidden_states, dim=-2)
hidden_states = orig_forward(hidden_states) # type: ignore
return hidden_states[..., :n, :]
norm_layer.forward = forward_ # type: ignore
return norm_layer
def register_shared_norm(
model: ModelPatcher,
share_group_norm: bool = True,
share_layer_norm: bool = True,
):
norm_layers = {"group": [], "layer": []}
get_norm_layers(model.model, norm_layers, share_layer_norm, share_group_norm)
print(
f"Patching {len(norm_layers['group'])} group norms, {len(norm_layers['layer'])} layer norms."
)
return [register_norm_forward(layer) for layer in norm_layers["group"]] + [
register_norm_forward(layer) for layer in norm_layers["layer"]
]
SHARE_NORM_OPTIONS = ["both", "group", "layer", "disabled"]
SHARE_ATTN_OPTIONS = ["q+k", "q+k+v", "disabled"]
class StyleAlignedSampleReferenceLatents:
@classmethod
def INPUT_TYPES(s):
return {"required":
{
"reference_image": ("IMAGE",),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING", ),
"model": ("MODEL",),
"vae": ("VAE", ),
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
"steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
"cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}),
"scheduler": (comfy.samplers.KSampler.SCHEDULERS.reverse(), ),
"denoise": ("FLOAT", {"default": 1, "min": 0.0, "max": 1.0, "step": 0.01}),
}
}
RETURN_TYPES = ("STEP_LATENTS","LATENT")
RETURN_NAMES = ("ref_latents", "noised_output")
FUNCTION = "run"
# CATEGORY = "style_aligned"
CATEGORY = "♾️Mixlab/Style"
def run(self, reference_image, positive, negative, model, vae, seed, steps, cfg,scheduler,denoise):
# TODO noise_mask?
def vae_encode_crop_pixels(pixels):
x = (pixels.shape[1] // 8) * 8
y = (pixels.shape[2] // 8) * 8
if pixels.shape[1] != x or pixels.shape[2] != y:
x_offset = (pixels.shape[1] % 8) // 2
y_offset = (pixels.shape[2] % 8) // 2
pixels = pixels[:, x_offset:x + x_offset, y_offset:y + y_offset, :]
return pixels
pixels=vae_encode_crop_pixels(reference_image)
t = vae.encode(pixels[:,:,:,:3])
latent_image = {"samples":t}
noise_seed=seed
sampler_name="ddim"
sampler = comfy.samplers.sampler_object(sampler_name)
total_steps = steps
if denoise < 1.0:
total_steps = int(steps/denoise)
comfy.model_management.load_models_gpu([model])
sigmas = comfy.samplers.calculate_sigmas_scheduler(model.model, scheduler, total_steps).cpu()
sigmas = sigmas[-(steps + 1):]
sigmas = sigmas.flip(0)
if sigmas[0] == 0:
sigmas[0] = 0.0001
latent = latent_image
latent_image = latent["samples"]
noise = torch.zeros(latent_image.size(), dtype=latent_image.dtype, layout=latent_image.layout, device="cpu")
noise_mask = None
if "noise_mask" in latent:
noise_mask = latent["noise_mask"]
ref_latents = []
def callback(step: int, x0: T, x: T, steps: int):
ref_latents.insert(0, x[0])
disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image, noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise_seed)
out = latent.copy()
out["samples"] = samples
out_noised = out
ref_latents = torch.stack(ref_latents)
return (ref_latents, out_noised)
class StyleAlignedReferenceSampler:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"ref_latents": ("STEP_LATENTS",),
"reference_image_text": ("STRING", {"multiline": True}),
"model": ("MODEL",),
"clip": ("CLIP", ),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING",),
"share_norm": (SHARE_NORM_OPTIONS,),
"share_attn": (SHARE_ATTN_OPTIONS,),
"scale": ("FLOAT", {"default": 1, "min": 0, "max": 2.0, "step": 0.01}),
"batch_size": ("INT", {"default": 2, "min": 1, "max": 8, "step": 1}),
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
"steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
"cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}),
"scheduler": (comfy.samplers.KSampler.SCHEDULERS, ),
"denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
},
}
RETURN_TYPES = ("LATENT", "LATENT")
RETURN_NAMES = ("output", "denoised_output")
FUNCTION = "patch"
# CATEGORY = "style_aligned"
CATEGORY = "♾️Mixlab/Style"
def patch(
self,
ref_latents,
reference_image_text,
model,
clip,
positive,
negative,
share_norm,
share_attn,
scale,
batch_size,
seed,steps,cfg,scheduler,denoise
) -> "tuple[dict, dict]":
m = model.clone()
# ref_latents = vae.encode(reference_image[:,:,:,:3])
tokens = clip.tokenize(reference_image_text)
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
ref_positive=[[cond, {"pooled_output": pooled}]]
noise_seed=seed
total_steps = steps
if denoise < 1.0:
total_steps = int(steps/denoise)
# comfy.model_management.load_models_gpu([model])
sigmas = comfy.samplers.calculate_sigmas_scheduler(model.model, scheduler, total_steps).cpu()
sigmas = sigmas[-(steps + 1):]
sampler_name="ddim"
sampler = comfy.samplers.sampler_object(sampler_name)
args = StyleAlignedArgs(share_attn)
# Concat batch with style latent
style_latent_tensor = ref_latents[0].unsqueeze(0)
height, width = style_latent_tensor.shape[-2:]
latent_t = torch.zeros(
[batch_size, 4, height, width], device=ref_latents.device
)
latent = {"samples": latent_t}
noise = comfy.sample.prepare_noise(latent_t, noise_seed)
latent_t = torch.cat((style_latent_tensor, latent_t), dim=0)
ref_noise = torch.zeros_like(noise[0]).unsqueeze(0)
noise = torch.cat((ref_noise, noise), dim=0)
x0_output = {}
preview_callback = latent_preview.prepare_callback(m, sigmas.shape[-1] - 1, x0_output)
# Replace first latent with the corresponding reference latent after each step
def callback(step: int, x0: T, x: T, steps: int):
preview_callback(step, x0, x, steps)
if (step + 1 < steps):
# 当ref_latents的step不够时
if step+1>len(ref_latents)-1:
step=len(ref_latents)-2
x[0] = ref_latents[step+1]
x0[0] = ref_latents[step+1]
# Register shared norms
share_group_norm = share_norm in ["group", "both"]
share_layer_norm = share_norm in ["layer", "both"]
register_shared_norm(m, share_group_norm, share_layer_norm)
# Patch cross attn
m.set_model_attn1_patch(SharedAttentionProcessor(args, scale))
# Add reference conditioning to batch
batched_condition = []
for i,condition in enumerate(positive):
additional = condition[1].copy()
batch_with_reference = torch.cat([ref_positive[i][0], condition[0].repeat([batch_size] + [1] * len(condition[0].shape[1:]))], dim=0)
if 'pooled_output' in additional and 'pooled_output' in ref_positive[i][1]:
# combine pooled output
pooled_output = torch.cat([ref_positive[i][1]['pooled_output'], additional['pooled_output'].repeat([batch_size]
+ [1] * len(additional['pooled_output'].shape[1:]))], dim=0)
additional['pooled_output'] = pooled_output
if 'control' in additional:
if 'control' in ref_positive[i][1]:
# combine control conditioning
control_hint = torch.cat([ref_positive[i][1]['control'].cond_hint_original, additional['control'].cond_hint_original.repeat([batch_size]
+ [1] * len(additional['control'].cond_hint_original.shape[1:]))], dim=0)
cloned_controlnet = additional['control'].copy()
cloned_controlnet.set_cond_hint(control_hint, strength=additional['control'].strength, timestep_percent_range=additional['control'].timestep_percent_range)
additional['control'] = cloned_controlnet
else:
# add zeros for first in batch
control_hint = torch.cat([torch.zeros_like(additional['control'].cond_hint_original), additional['control'].cond_hint_original.repeat([batch_size]
+ [1] * len(additional['control'].cond_hint_original.shape[1:]))], dim=0)
cloned_controlnet = additional['control'].copy()
cloned_controlnet.set_cond_hint(control_hint, strength=additional['control'].strength, timestep_percent_range=additional['control'].timestep_percent_range)
additional['control'] = cloned_controlnet
batched_condition.append([batch_with_reference, additional])
disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
samples = comfy.sample.sample_custom(
m,
noise,
cfg,
sampler,
sigmas,
batched_condition,
negative,
latent_t,
callback=callback,
disable_pbar=disable_pbar,
seed=noise_seed,
)
# remove reference image
samples = samples[1:]
out = latent.copy()
out["samples"] = samples
if "x0" in x0_output:
out_denoised = latent.copy()
x0 = x0_output["x0"][1:]
out_denoised["samples"] = m.model.process_latent_out(x0.cpu())
else:
out_denoised = out
return (out, out_denoised)
class StyleAlignedBatchAlign:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"share_norm": (SHARE_NORM_OPTIONS,),
"share_attn": (SHARE_ATTN_OPTIONS,),
"scale": ("FLOAT", {"default": 1, "min": 0, "max": 1.0, "step": 0.1}),
}
}
RETURN_TYPES = ("MODEL",)
FUNCTION = "patch"
# CATEGORY = "style_aligned"
CATEGORY = "♾️Mixlab/Style"
def patch(
self,
model: ModelPatcher,
share_norm: str,
share_attn: str,
scale: float,
):
m = model.clone()
share_group_norm = share_norm in ["group", "both"]
share_layer_norm = share_norm in ["layer", "both"]
register_shared_norm(model, share_group_norm, share_layer_norm)
args = StyleAlignedArgs(share_attn)
m.set_model_attn1_patch(SharedAttentionProcessor(args, scale))
return (m,)
+17 -141
View File
@@ -12,8 +12,6 @@ import comfy.utils
# import numpy as np
import torch
import random
from lark import Lark, Transformer, v_args
global _available
_available=True
@@ -64,14 +62,7 @@ except:
def translate(text):
global text_pipe,zh_en_model,zh_en_tokenizer
if zh_en_model==None:
zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path,padding=True, truncation=True)
zh_en_model.to("cuda" if torch.cuda.is_available() else "cpu")
def translate(zh_en_tokenizer,zh_en_model,text):
with torch.no_grad():
encoded = zh_en_tokenizer([text], return_tensors="pt")
encoded.to(zh_en_model.device)
@@ -111,24 +102,18 @@ def text_generate(text_pipe,input,seed=None):
import re
def correct_prompt_syntax(prompt=""):
def correct_prompt_syntax(prompt):
# print("input prompt",prompt)
print("input prompt",prompt)
corrected_elements = []
# 处理成统一的英文标点
prompt = prompt.replace('(', '(').replace(')', ')').replace(',', ',').replace(';', ',').replace('。', '.').replace(':',':')
# 删除多余的空格
prompt = re.sub(r'\s+', ' ', prompt).strip()
prompt = prompt.replace("< ","<").replace(" >",">").replace("( ","(").replace(" )",")").replace("[ ","[").replace(' ]',']')
# 分词
prompt_elements = prompt.split(',')
def balance_brackets(element, open_bracket, close_bracket):
open_brackets_count = element.count(open_bracket)
close_brackets_count = element.count(close_bracket)
return element + close_bracket * (open_brackets_count - close_brackets_count)
for element in prompt_elements:
element = element.strip()
@@ -148,118 +133,21 @@ def correct_prompt_syntax(prompt=""):
corrected_elements.append(corrected_element)
# 重组修正后的prompt
return ','.join(corrected_elements)
corrected_prompt = ', '.join(corrected_elements)
print("output prompt",corrected_prompt)
return corrected_prompt
def balance_brackets(element, open_bracket, close_bracket):
open_brackets_count = element.count(open_bracket)
close_brackets_count = element.count(close_bracket)
return element + close_bracket * (open_brackets_count - close_brackets_count)
# # 示例使用
# test_prompt = "((middle-century castles)), [forsaken: 0.8], (mystery dragons: 1.3, mist forests, sunsets, quiet; (((dummy)), [fisting city: 0.5] background, radiant, soft and flavoured,] promising mountains, ((starry: 1.6), [[crowds], [middle-century castle: urban landscapes of the future: 0.5], [yellow: bright sun: 0.7], overlooking"
# corrected_prompt = correct_prompt_syntax(test_prompt)
# print(corrected_prompt)
def detect_language(input_str):
# 统计中文和英文字符的数量
count_cn = count_en = 0
for char in input_str:
if '\u4e00' <= char <= '\u9fff':
count_cn += 1
elif char.isalpha():
count_en += 1
# 根据统计的字符数量判断主要语言
if count_cn > count_en:
return "cn"
elif count_en > count_cn:
return "en"
else:
return "unknow"
#定义Prompt文法
grammar = """
start: sentence
sentence: phrase ("," phrase)*
phrase: emphasis | weight | word | lora | embedding | schedule
emphasis: "(" sentence ")" -> emphasis
| "[" sentence "]" -> weak_emphasis
weight: "(" word ":" NUMBER ")"
schedule: "[" word ":" word ":" NUMBER "]"
lora: "<" WORD ":" WORD (":" NUMBER)? (":" NUMBER)? ">"
embedding: "embedding" ":" WORD (":" NUMBER)? (":" NUMBER)?
word: WORD
NUMBER: /\s*-?\d+(\.\d+)?\s*/
WORD: /[^,:\(\)\[\]<>]+/
"""
@v_args(inline=True) # Decorator to flatten the tree directly into the function arguments
class ChinesePromptTranslate(Transformer):
def sentence(self, *args):
return ", ".join(args)
def phrase(self, *args):
return "".join(args)
def emphasis(self, *args):
# Reconstruct the emphasis with translated content
return "(" + "".join(args) + ")"
def weak_emphasis(self, *args):
print('weak_emphasis:',args)
return "[" + "".join(args) + "]"
def embedding(self,*args):
print('prompt embedding',args[0])
if len(args) == 1:
# print('prompt embedding',str(args[0]))
# 只传递了一个参数,意味着只有embedding名称没有数字
embedding_name = str(args[0])
return f"embedding:{embedding_name}"
elif len(args) > 1:
embedding_name,*numbers = args
if len(numbers)==2:
return f"embedding:{embedding_name}:{numbers[0]}:{numbers[1]}"
elif len(numbers)==1:
return f"embedding:{embedding_name}:{numbers[0]}"
else:
return f"embedding:{embedding_name}"
def lora(self,*args):
print('lora prompt',*args)
if len(args) == 1:
return f"<lora:{loar_name}>"
elif len(args) > 1:
# print('lora', args)
_,loar_name,*numbers = args
loar_name = str(loar_name).strip()
if len(numbers)==2:
return f"<lora:{loar_name}:{numbers[0]}:{numbers[1]}>"
elif len(numbers)==1:
return f"<lora:{loar_name}:{numbers[0]}>"
else:
return f"<lora:{loar_name}>"
def weight(self, word,number):
translated_word = translate(str(word)).rstrip('.')
return f"({translated_word}:{str(number).strip()})"
def schedule(self,*args):
print('prompt schedule',args)
data = [str(arg).strip() for arg in args]
return f"[{':'.join(data)}]"
def word(self, word):
# Translate each word using the dictionary
if detect_language(str(word)) == "cn":
return translate(str(word)).rstrip('.')
else:
return str(word).rstrip('.')
class ChinesePrompt:
@@ -297,16 +185,16 @@ class ChinesePrompt:
zh_en_tokenizer=None
def run(self,text,seed,generation):
global text_pipe,zh_en_model,zh_en_tokenizer
seed=seed[0]
generation=generation[0]
# 进度条
pbar = comfy.utils.ProgressBar(len(text)+1)
texts = [correct_prompt_syntax(t) for t in text]
global text_pipe,zh_en_model,zh_en_tokenizer
print('correct_prompt_syntax::',texts)
if zh_en_model==None:
zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path,padding=True, truncation=True)
@@ -322,16 +210,9 @@ class ChinesePrompt:
# print('zh_en_model device',zh_en_model.device,text_pipe.model.device,torch.cuda.current_device() )
en_texts=[]
for t in texts:
if t:
# translated_text = translated_word = translate(zh_en_tokenizer,zh_en_model,str(t))
parser = Lark(grammar, start="start", parser="lalr", transformer=ChinesePromptTranslate())
# print('t',t)
result = parser.parse(t).children
# print('en_result',result)
# en_text=translate(zh_en_tokenizer,zh_en_model,text_without_syntax)
en_texts.append(result[0])
en_text=translate(zh_en_tokenizer,zh_en_model,t)
en_texts.append(en_text)
zh_en_model.to('cpu')
print("test en_text",en_texts)
@@ -351,11 +232,8 @@ class ChinesePrompt:
pbar.update(1)
text_pipe.model.to('cpu')
print('prompt_result',prompt_result,)
# prompt_result = [','.join(correct_prompt_syntax(p)) for p in prompt_result]
if len(prompt_result)==0:
prompt_result=[""]
prompt_result = [correct_prompt_syntax(p) for p in prompt_result]
return {
"ui":{
"prompt": prompt_result
@@ -363,8 +241,6 @@ class ChinesePrompt:
"result": (prompt_result,)}
class PromptGenerate:
global _available
+32 -281
View File
@@ -1,25 +1,12 @@
import os,platform
import re,random,json
from PIL import Image
import numpy as np
# FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
import folder_paths
import matplotlib.font_manager as fm
# import matplotlib.font_manager as fm
import torch
import importlib.util
def create_incrementing_list(min_value, max_value, step, count):
l1 = [int(min_value + i * step) for i in range(count) if min_value + i * step <= max_value]
l2 = [float(min_value + i * step) for i in range(count) if min_value + i * step <= max_value]
return (l1,l2)
def split_list(lst, chunk_size, transition_size):
result = []
for i in range(0, len(lst), chunk_size):
start = i - transition_size
end = i + chunk_size + transition_size
result.append(lst[max(start, 0):end])
return result
def recursive_search(directory, excluded_dir_names=None):
if not os.path.isdir(directory):
@@ -77,37 +64,7 @@ def get_system_font_path():
# return hash_value
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
def create_temp_file(image):
output_dir = folder_paths.get_temp_directory()
(
full_output_folder,
filename,
counter,
subfolder,
_,
) = folder_paths.get_save_image_path('tmp', output_dir)
im=tensor2pil(image)
image_file = f"{filename}_{counter:05}.png"
image_path=os.path.join(full_output_folder, image_file)
im.save(image_path,compress_level=4)
return [{
"filename": image_file,
"subfolder": subfolder,
"type": "temp"
}]
def get_font_files(directory):
font_files = {}
@@ -158,6 +115,7 @@ class ColorInput:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"color":("TCOLOR",),
},
}
@@ -167,7 +125,7 @@ class ColorInput:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Color"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,False,False,False,False,)
@@ -196,7 +154,7 @@ class FontInput:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -229,7 +187,7 @@ class TextToNumber:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Text"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -284,10 +242,10 @@ class FloatSlider:
}
RETURN_TYPES = ("FLOAT",)
RETURN_NAMES = ('weight(0-1)',)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -340,7 +298,7 @@ class IntNumber:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -358,7 +316,7 @@ class MultiplicationNode:
return {"required": {
"numberA":(any_type,),
"multiply_by":("FLOAT", {
"default": 1,
"default": 0,
"min": -2, #Minimum value
"max": 0xffffffffffffffff,
"step": 0.01, #Slider's step
@@ -392,7 +350,7 @@ class TextInput:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"text": ("STRING",{"multiline": True,"default": ""})
"text": ("STRING",{"multiline": True,"default": ""}),
},
}
@@ -400,7 +358,7 @@ class TextInput:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -409,61 +367,6 @@ class TextInput:
return (text,)
class IncrementingListNode:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"min_value": ("FLOAT", {
"default": 0,
"min": -2000, #Minimum value
"max": 0xffffffffffffffff,
"step": 0.01, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"max_value": ("FLOAT", {
"default": 10,
"min": -2000, #Minimum value
"max": 0xffffffffffffffff,
"step": 0.01, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"step": ("FLOAT", {
"default": 0,
"min": -2000, #Minimum value
"max": 0xffffffffffffffff,
"step": 0.01, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
}),
"count": ("INT", {
"default": 1,
"min": 1, #Minimum value
"max": 0xffffffffffffffff,
"step":1, #Slider's step
"display": "number" # Cosmetic only: display as "number" or "slider"
})
},
"optional":{
"seed":("INT", {"default": -1, "min": -1, "max": 1000000}),
},
}
RETURN_TYPES = ("INT","FLOAT",)
RETURN_NAMES = ('int_list','float_list',)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Video"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (True,True,)
def run(self,min_value,max_value,step,count,seed):
print('create_incrementing_list',seed)
l1,l2=create_incrementing_list(min_value,max_value,step,count)
return (l1,l2,)
# 接收一个值,然后根据字符串或数值长度计算延迟时间,用户可以自定义延迟"字/s",延迟之后将转化
import comfy.samplers
@@ -479,6 +382,7 @@ class AnyType(str):
return False
any_type = AnyType("*")
import time
class DynamicDelayProcessor:
@@ -556,69 +460,6 @@ class DynamicDelayProcessor:
# app 配置节点
class AppInfo:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"name": ("STRING",{"multiline": False,"default": "Mixlab-App","dynamicPrompts": False}),
"input_ids":("STRING",{"multiline": True,"default": "\n".join(["1","2","3"]),"dynamicPrompts": False}),
"output_ids":("STRING",{"multiline": True,"default": "\n".join(["5","9"]),"dynamicPrompts": False}),
},
"optional":{
"IMAGE": ("IMAGE",),
"description":("STRING",{"multiline": True,"default": "","dynamicPrompts": False}),
"version":("INT", {
"default": 1,
"min": 1,
"max": 10000,
"step": 1,
"display": "number"
}),
"share_prefix":("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
"link":("STRING",{"multiline": False,"default": "https://","dynamicPrompts": False}),
"category":("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
"auto_save": (["enable","disable"],),
}
}
RETURN_TYPES = ()
# RETURN_NAMES = ("IMAGE",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab"
OUTPUT_NODE = True
INPUT_IS_LIST = True
# OUTPUT_IS_LIST = (True,)
def run(self,name,input_ids,output_ids,IMAGE,description,version,share_prefix,link,category,auto_save):
name=name[0]
im=None
if IMAGE:
im=IMAGE[0][0]
#TODO batch 的方式需要处理
im=create_temp_file(im)
# image [img,] img[batch,w,h,a] 列表里面是batch,
input_ids=input_ids[0]
output_ids=output_ids[0]
description=description[0]
version=version[0]
share_prefix=share_prefix[0]
link=link[0]
category=category[0]
# id=get_json_hash([name,im,input_ids,output_ids,description,version])
return {"ui": {"json": [name,im,input_ids,output_ids,description,version,share_prefix,link,category]}, "result": ()}
@@ -626,11 +467,9 @@ class SwitchByIndex:
@classmethod
def INPUT_TYPES(cls):
return {
"optional":{
"A":(any_type,),
"B":(any_type,),
},
"required": {
"required": {
"A":(any_type,),
"B":(any_type,),
"index":("INT", {
"default": -1,
"min": -1,
@@ -642,17 +481,17 @@ class SwitchByIndex:
}
}
RETURN_TYPES = (any_type,"INT",)
RETURN_NAMES = ("list", "count",)
RETURN_TYPES = (any_type,)
RETURN_NAMES = ("C",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True, False,)
OUTPUT_IS_LIST = (True,)
def run(self, A=[],B=[],index=-1,flat='on'):
def run(self, A,B,index,flat):
flat=flat[0]
@@ -671,43 +510,10 @@ class SwitchByIndex:
try:
C=[C[index]]
except Exception as e:
C=[C[-1]] #最后一个
C=[]
return (C,)
return (C, len(C),)
class ListSplit:
@classmethod
def INPUT_TYPES(cls):
return {
"optional":{
"A":(any_type,),
},
"required": {
"chunk_size": ("INT", {"default": 10, "min": 1, "step": 1}),
"transition_size": ("INT", {"default": 0, "min": 0, "step": 1}),
"index": ("INT", {"default": -1, "min": -1, "step": 1}),
}
}
RETURN_TYPES = (any_type,)
RETURN_NAMES = ("B",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,)
def run(self, A=[],chunk_size=[10],transition_size=[0],index=[-1]):
# print(len(A))
B=split_list(A,chunk_size[0],transition_size[0])
if index[0]>-1:
B=B[index[0]]
return (B,)
class LimitNumber:
@@ -738,7 +544,7 @@ class LimitNumber:
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Input"
CATEGORY = "♾️Mixlab/Utils"
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
@@ -796,85 +602,30 @@ class ListStatistics:
class TESTNODE_:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"ANY":(any_type,),
},
return {"required": { "ANY":(any_type,), },
}
RETURN_TYPES = (any_type,)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Test"
CATEGORY = "♾️Mixlab/__TEST"
OUTPUT_NODE = True
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,)
def run(self,ANY):
print(type(ANY))
try:
print(ANY[0].shape)
img= tensor2pil(ANY[0])
print(img.size)
except:
print('')
print(ANY)
# data=ANY
list_stats = ListStatistics()
# 调用count_types方法进行统计
result = list_stats.count_types(ANY)
# 假设我们有一个模块文件名为 my_module.py,它位于 'importables' 目录下
module_path = os.path.join(os.path.dirname(__file__),'test.py')
# 使用 spec_from_file_location 获取模块的元数据(名称、定义等)
spec = importlib.util.spec_from_file_location('test', module_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
functions = getattr(module, 'run') # 获取函数
functions(ANY)
return {"ui": {"data": result,"type":[str(type(ANY[0]))]}, "result": (ANY,)}
class TESTNODE_TOKEN:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"text":("STRING", {"forceInput": True,}),
"clip": ("CLIP", )
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Test"
OUTPUT_NODE = True
INPUT_IS_LIST = False
OUTPUT_IS_LIST = (False,)
def run(self,text,clip=None):
# print(text)
tokens = clip.tokenize(text)
tokens=[v for v in tokens.values()][0][0]
tokens=json.dumps(tokens)
return (tokens,)
class CreateSeedNode:
def __init__(self):
@@ -894,7 +645,7 @@ class CreateSeedNode:
OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Experiment"
CATEGORY = "♾️Mixlab/Utils"
def run(self, seed):
return (seed,)
@@ -921,7 +672,7 @@ class CreateCkptNames:
# OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Experiment"
CATEGORY = "♾️Mixlab/Utils"
def run(self, ckpt_names):
ckpt_names=ckpt_names.split('\n')
@@ -950,7 +701,7 @@ class CreateLoraNames:
# OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Experiment"
CATEGORY = "♾️Mixlab/Utils"
def run(self, lora_names):
lora_names=lora_names.split('\n')
@@ -981,7 +732,7 @@ class CreateSampler_names:
# OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Experiment"
CATEGORY = "♾️Mixlab/Utils"
def run(self, sampler_names):
sampler_names=sampler_names.split('\n')
+179
View File
@@ -0,0 +1,179 @@
# https://github.com/openai/consistencydecoder/blob/main/consistencydecoder/__init__.py
import folder_paths
from comfy import model_management
import math
import torch
import numpy as np
from PIL import Image
class ConsistencyDecoderWrapper:
def __init__(self, decoder):
self.decoder = decoder
def decode(self, x):
return self.decoder(x)
def _extract_into_tensor(arr, timesteps, broadcast_shape):
# from: https://github.com/openai/guided-diffusion/blob/22e0df8183507e13a7813f8d38d51b072ca1e67c/guided_diffusion/gaussian_diffusion.py#L895 """
res = arr[timesteps].float()
dims_to_append = len(broadcast_shape) - len(res.shape)
return res[(...,) + (None,) * dims_to_append]
def betas_for_alpha_bar(num_diffusion_timesteps, alpha_bar, max_beta=0.999):
# from: https://github.com/openai/guided-diffusion/blob/22e0df8183507e13a7813f8d38d51b072ca1e67c/guided_diffusion/gaussian_diffusion.py#L45
betas = []
for i in range(num_diffusion_timesteps):
t1 = i / num_diffusion_timesteps
t2 = (i + 1) / num_diffusion_timesteps
betas.append(min(1 - alpha_bar(t2) / alpha_bar(t1), max_beta))
return torch.tensor(betas)
class ConsistencyDecoder:
def __init__(self, device="cuda:0", download_target=""):
self.n_distilled_steps = 64
# download_target = _download("https://openaipublic.azureedge.net/diff-vae/c9cebd3132dd9c42936d803e33424145a748843c8f716c0814838bdc8a2fe7cb/decoder.pt", download_root)
self.ckpt = torch.jit.load(download_target).to(device)
self.device = device
sigma_data = 0.5
betas = betas_for_alpha_bar(
1024, lambda t: math.cos((t + 0.008) / 1.008 * math.pi / 2) ** 2
).to(device)
alphas = 1.0 - betas
alphas_cumprod = torch.cumprod(alphas, dim=0)
self.sqrt_alphas_cumprod = torch.sqrt(alphas_cumprod)
self.sqrt_one_minus_alphas_cumprod = torch.sqrt(1.0 - alphas_cumprod)
sqrt_recip_alphas_cumprod = torch.sqrt(1.0 / alphas_cumprod)
sigmas = torch.sqrt(1.0 / alphas_cumprod - 1)
self.c_skip = (
sqrt_recip_alphas_cumprod
* sigma_data**2
/ (sigmas**2 + sigma_data**2)
)
self.c_out = sigmas * sigma_data / (sigmas**2 + sigma_data**2) ** 0.5
self.c_in = sqrt_recip_alphas_cumprod / (sigmas**2 + sigma_data**2) ** 0.5
@staticmethod
def round_timesteps(
timesteps, total_timesteps, n_distilled_steps, truncate_start=True
):
with torch.no_grad():
space = torch.div(total_timesteps, n_distilled_steps, rounding_mode="floor")
rounded_timesteps = (
torch.div(timesteps, space, rounding_mode="floor") + 1
) * space
if truncate_start:
rounded_timesteps[rounded_timesteps == total_timesteps] -= space
else:
rounded_timesteps[rounded_timesteps == total_timesteps] -= space
rounded_timesteps[rounded_timesteps == 0] += space
return rounded_timesteps
@staticmethod
def ldm_transform_latent(z, extra_scale_factor=1):
channel_means = [0.38862467, 0.02253063, 0.07381133, -0.0171294]
channel_stds = [0.9654121, 1.0440036, 0.76147926, 0.77022034]
if len(z.shape) != 4:
raise ValueError()
z = z * 0.18215
channels = [z[:, i] for i in range(z.shape[1])]
channels = [
extra_scale_factor * (c - channel_means[i]) / channel_stds[i]
for i, c in enumerate(channels)
]
return torch.stack(channels, dim=1)
@torch.no_grad()
def __call__(
self,
features: torch.Tensor,
schedule=[1.0, 0.5],
):
features = self.ldm_transform_latent(features)
ts = self.round_timesteps(
torch.arange(0, 1024),
1024,
self.n_distilled_steps,
truncate_start=False,
)
shape = (
features.size(0),
3,
8 * features.size(2),
8 * features.size(3),
)
x_start = torch.zeros(shape, device=features.device, dtype=features.dtype)
schedule_timesteps = [int((1024 - 1) * s) for s in schedule]
for i in schedule_timesteps:
t = ts[i].item()
t_ = torch.tensor([t] * features.shape[0]).to(self.device)
noise = torch.randn_like(x_start)
x_start = (
_extract_into_tensor(self.sqrt_alphas_cumprod, t_, x_start.shape)
* x_start
+ _extract_into_tensor(
self.sqrt_one_minus_alphas_cumprod, t_, x_start.shape
)
* noise
)
c_in = _extract_into_tensor(self.c_in, t_, x_start.shape)
model_output = self.ckpt(c_in * x_start, t_, features=features)
B, C = x_start.shape[:2]
model_output, _ = torch.split(model_output, C, dim=1)
pred_xstart = (
_extract_into_tensor(self.c_out, t_, x_start.shape) * model_output
+ _extract_into_tensor(self.c_skip, t_, x_start.shape) * x_start
).clamp(-1, 1)
x_start = pred_xstart
return x_start
class VAELoader:
@classmethod
def INPUT_TYPES(s):
return {"required": { "vae_name": (folder_paths.get_filename_list("vae"), )}}
RETURN_TYPES = ("VAE",)
FUNCTION = "load_vae"
CATEGORY = "♾️Mixlab/__TEST"
#TODO: scale factor?
def load_vae(self, vae_name):
vae_path = folder_paths.get_full_path("vae", vae_name)
device = 'cuda:0'
# print('device',device)
consistencyDecoder = ConsistencyDecoder(device=device,
download_target=vae_path) # Model size: 2.49 GB
vae = ConsistencyDecoderWrapper(consistencyDecoder)
return (vae,)
class VAEDecode:
@classmethod
def INPUT_TYPES(s):
return {"required": { "samples": ("LATENT", ), "vae": ("VAE", )}}
RETURN_TYPES = ("IMAGE",)
FUNCTION = "decode"
CATEGORY = "♾️Mixlab/__TEST"
def decode(self, vae, samples):
image = vae.decode(samples["samples"].to("cuda:0"))
image = image[0].cpu().numpy()
image = (image + 1.0) * 127.5
image = image.clip(0, 255).astype(np.uint8)
image = Image.fromarray(image.transpose(1, 2, 0))
image = image.convert("RGB")
image = np.array(image).astype(np.float32) / 255.0
image = torch.from_numpy(image)[None,]
return (image, )
-692
View File
@@ -1,692 +0,0 @@
import os
import hashlib
import json
import subprocess
import shutil
import re
import time,math
import numpy as np
from typing import List
import torch
from PIL import Image, ImageOps
from PIL.PngImagePlugin import PngInfo
import cv2,random,string
from pathlib import Path
import folder_paths
from comfy.k_diffusion.utils import FolderOfImages
from comfy.utils import common_upscale
def generate_folder_name(directory,video_path):
# Get the directory and filename from the video path
_, filename = os.path.split(video_path)
# Generate a random string of lowercase letters and digits
random_string = ''.join(random.choices(string.ascii_lowercase + string.digits, k=8))
# Create the folder name by combining the random string and the filename
folder_name = random_string + '_' + filename
# Create the full folder path by joining the directory and the folder name
folder_path = os.path.join(directory, folder_name)
return folder_path
def create_folder(directory,video_path):
folder_path = generate_folder_name(directory,video_path)
os.makedirs(folder_path)
return folder_path
def split_video(video_path, video_segment_frames, transition_frames, output_dir):
# 读取视频文件
video_capture = cv2.VideoCapture(video_path)
# 获取视频的总帧数和帧率
total_frames = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
fps = video_capture.get(cv2.CAP_PROP_FPS)
# 计算每个视频片段的总帧数,包括过渡帧
segment_total_frames = video_segment_frames + transition_frames
# 计算可以分割的片段数量,向上取整
num_segments = (total_frames + transition_frames - 1) // segment_total_frames
vs=[]
# 计算每个片段的起始帧和结束帧
start_frame = 0
for i in range(num_segments):
# 计算当前片段的结束帧,注意最后一个片段可能没有过渡帧
end_frame = min(start_frame + segment_total_frames, total_frames)
# 打印当前片段的起始帧和结束帧
print(f"Segment {i+1}: Start Frame {start_frame}, End Frame {end_frame}")
# 保存当前片段为一个视频文件
segment_video_path = f"{output_dir}/segment_{i+1}.avi"
fourcc = cv2.VideoWriter_fourcc(*'XVID')
segment_video = cv2.VideoWriter(segment_video_path, fourcc, fps, (int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH)),
int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))))
for frame_num in range(start_frame, end_frame):
ret, frame = video_capture.read()
if ret:
segment_video.write(frame)
else:
break # 如果读取失败,则退出循环
# 更新起始帧为下一个片段的起始位置
start_frame = end_frame + transition_frames
vs.append(segment_video_path)
# 释放视频捕获对象
video_capture.release()
# print(vs)
return (vs,total_frames,fps)
folder_paths.folder_names_and_paths["video_formats"] = (
[
os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "video_formats"),
],
[".json"]
)
ffmpeg_path = shutil.which("ffmpeg")
if ffmpeg_path is None:
print("ffmpeg could not be found. Using ffmpeg from imageio-ffmpeg.")
from imageio_ffmpeg import get_ffmpeg_exe
try:
ffmpeg_path = get_ffmpeg_exe()
except:
print("ffmpeg could not be found. Outputs that require it have been disabled")
# Tensor to PIL
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
# Convert PIL to Tensor
def pil2tensor(image):
return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
def count_files(directory):
count = 0
for root, dirs, files in os.walk(directory):
count += len(files)
return count
def create_temp_file(image):
output_dir = folder_paths.get_temp_directory()
c=count_files(output_dir)
(
full_output_folder,
filename,
counter,
subfolder,
_,
) = folder_paths.get_save_image_path('temp_', output_dir)
image=tensor2pil(image)
image_file = f"{filename}_{c}_{counter:05}.png"
image_path=os.path.join(full_output_folder, image_file)
image.save(image_path,compress_level=4)
return [{
"filename": image_file,
"subfolder": subfolder,
"type": "temp"
}]
def split_list(lst, chunk_size, transition_size):
result = []
for i in range(0, len(lst), chunk_size):
start = i - transition_size
end = i + chunk_size + transition_size
result.append(lst[max(start, 0):end])
return result
# images = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
# chunk_size = 3
# transition_size = 1
# result = split_list(images, chunk_size, transition_size)
# print(result)
class ImageListReplace:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"images": ("IMAGE",),
"start_index":("INT", {"default": 0, "min": 0, "step": 1}),
"end_index":("INT", {"default": 0, "min": 0, "step": 1}),
"invert": ("BOOLEAN", {"default": False}),
},
"optional":{
"image_replace": ("IMAGE",),
"images_replace": ("IMAGE",),
}
}
RETURN_TYPES = ("IMAGE","IMAGE",)
RETURN_NAMES = ("images","select_images",)
FUNCTION = "run"
CATEGORY = "♾️Mixlab/Video"
OUTPUT_NODE = True
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,True,)
def run(self, images,start_index=[0],end_index=[0],invert=[False],image_replace=None,images_replace=None):
start_index=start_index[0]
end_index=end_index[0]
invert=invert[0]
image_rs=[]
if image_replace!=None:
for i in range(end_index-start_index+1):
image_rs.append(image_replace[0])
if images_replace!=None:
image_rs=images_replace
# 如果image replace 为空
if image_replace==None and images_replace==None:
# print('如果image replace 为空',images[0])
# [[tensor(
# tensor([[[[0.
first_image=tensor2pil(images[0][0])
width, height = first_image.size
image_replace=Image.new("RGB", (width, height), (0, 0, 0))
image_replace=pil2tensor(image_replace)
for i in range(end_index-start_index+1):
image_rs.append(image_replace)
new_images=[]
select_images=[]
k=0
for i in range(len(images)):
if i>=start_index and i<=end_index:
if invert:
new_images.append(images[i])
else:
new_images.append(image_rs[k])
select_images.append(images[i])
k+=1
else:
if invert:
new_images.append(image_rs[k])
select_images.append(images[i])
k+=1
else:
new_images.append(images[i])
imss=[]
# print(len(images))
for i in range(len(images)):
t=images[i][0]
t=tensor2pil(t)
t = t.convert("RGB")
original_width, original_height = t.size
scale = 300 / original_width
new_height = int(original_height * scale)
t = t.resize((300, new_height))
ims=create_temp_file(pil2tensor(t))
imss.append(ims[0])
# image_replace=create_temp_file(image_replace)
return {"ui":{"_images": imss},"result": (new_images,select_images,)}
# The code is based on ComfyUI-VideoHelperSuite modification.
class LoadVideoAndSegment:
@classmethod
def INPUT_TYPES(s):
video_extensions = ['webm', 'mp4', 'mkv', 'gif']
input_dir = folder_paths.get_input_directory()
files = []
for f in os.listdir(input_dir):
if os.path.isfile(os.path.join(input_dir, f)):
file_parts = f.split('.')
if len(file_parts) > 1 and (file_parts[-1] in video_extensions):
files.append(f)
return {"required": {
"video": (sorted(files), {"video_upload": True}),
"video_segment_frames": ("INT", {"default": 10, "min": 1, "step": 1}),
"transition_frames": ("INT", {"default": 0, "min": 0, "step": 1}),
},}
CATEGORY = "♾️Mixlab/Video"
RETURN_TYPES = ("SCENE_VIDEO","INT", "INT","INT",)
RETURN_NAMES = ("scenes_video","scenes_count","frame_count","fps",)
FUNCTION = "load_video"
OUTPUT_NODE = True
OUTPUT_IS_LIST = (True,False,False,False,)
def is_gif(self, filename):
file_parts = filename.split('.')
return len(file_parts) > 1 and file_parts[-1] == "gif"
def load_video_cv_fallback(self, video, frame_load_cap, skip_first_frames):
try:
video_cap = cv2.VideoCapture(folder_paths.get_annotated_filepath(video))
if not video_cap.isOpened():
raise ValueError(f"{video} could not be loaded with cv fallback.")
# set video_cap to look at start_index frame
images = []
total_frame_count = 0
frames_added = 0
base_frame_time = 1/video_cap.get(cv2.CAP_PROP_FPS)
target_frame_time = base_frame_time
time_offset=0.0
while video_cap.isOpened():
if time_offset < target_frame_time:
is_returned, frame = video_cap.read()
# if didn't return frame, video has ended
if not is_returned:
break
time_offset += base_frame_time
if time_offset < target_frame_time:
continue
time_offset -= target_frame_time
# if not at start_index, skip doing anything with frame
total_frame_count += 1
if total_frame_count <= skip_first_frames:
continue
# TODO: do whatever operations need to happen, like force_size, etc
# opencv loads images in BGR format (yuck), so need to convert to RGB for ComfyUI use
# follow up: can videos ever have an alpha channel?
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
# convert frame to comfyui's expected format (taken from comfy's load image code)
image = Image.fromarray(frame)
image = ImageOps.exif_transpose(image)
image = np.array(image, dtype=np.float32) / 255.0
image = torch.from_numpy(image)[None,]
images.append(image)
frames_added += 1
# if cap exists and we've reached it, stop processing frames
if frame_load_cap > 0 and frames_added >= frame_load_cap:
break
finally:
video_cap.release()
images = torch.cat(images, dim=0)
return (images, frames_added)
def load_video(self, video,video_segment_frames,transition_frames ):
video_path = folder_paths.get_annotated_filepath(video)
# check if video is a gif - will need to use cv fallback to read frames
# use cv fallback if ffmpeg not installed or gif
# if ffmpeg_path is None:
# return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
# otherwise, continue with ffmpeg
# args_dummy = [ffmpeg_path, "-i", video_path, "-f", "null", "-"]
# try:
# with subprocess.Popen(args_dummy, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) as proc:
# for line in proc.stderr.readlines():
# match = re.search(", ([1-9]|\\d{2,})x(\\d+)",line.decode('utf-8'))
# if match is not None:
# size = [int(match.group(1)), int(match.group(2))]
# break
# except Exception as e:
# print(f"Retrying with opencv due to ffmpeg error: {e}")
# return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
# args_all_frames = [ffmpeg_path, "-i", video_path, "-v", "error",
# "-pix_fmt", "rgb24"]
# vfilters = []
# if skip_first_frames > 0:
# vfilters.append(f"select=gt(n\\,{skip_first_frames-1})")
# if frame_load_cap > 0:
# vfilters.append(f"select=gt({frame_load_cap}\\,n)")
# #manually calculate aspect ratio to ensure reads remain aligned
# if len(vfilters) > 0:
# args_all_frames += ["-vf", ",".join(vfilters)]
# args_all_frames += ["-f", "rawvideo", "-"]
# images = []
# try:
# with subprocess.Popen(args_all_frames, stdout=subprocess.PIPE) as proc:
# #Manually buffer enough bytes for an image
# bpi = size[0]*size[1]*3
# current_bytes = bytearray(bpi)
# current_offset=0
# while True:
# bytes_read = proc.stdout.read(bpi - current_offset)
# if bytes_read is None:#sleep to wait for more data
# time.sleep(.2)
# continue
# if len(bytes_read) == 0:#EOF
# break
# current_bytes[current_offset:len(bytes_read)] = bytes_read
# current_offset+=len(bytes_read)
# if current_offset == bpi:
# images.append(np.array(current_bytes, dtype=np.float32).reshape(size[1], size[0], 3) / 255.0)
# current_offset = 0
# except Exception as e:
# print(f"Retrying with opencv due to ffmpeg error: {e}")
# return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
# imgs=split_list(images,video_segment_frames,transition_frames)
# temp path
tp=folder_paths.get_temp_directory()
basename = os.path.basename(video_path) # 获取文件名
name_without_extension = os.path.splitext(basename)[0] # 去掉文件后缀
folder_path = create_folder(tp,name_without_extension)
# 导出的数据
scenes_video,total_frames,fps=split_video(video_path,video_segment_frames,
transition_frames,folder_path)
# imgs=[torch.from_numpy(np.stack(im)) for im in imgs]
# images = torch.from_numpy(np.stack(images))
return (scenes_video,len(scenes_video), total_frames,fps,)
@classmethod
def IS_CHANGED(s, video, **kwargs):
image_path = folder_paths.get_annotated_filepath(video)
m = hashlib.sha256()
with open(image_path, 'rb') as f:
m.update(f.read())
return m.digest().hex()
@classmethod
def VALIDATE_INPUTS(s, video, **kwargs):
if not folder_paths.exists_annotated_filepath(video):
return "Invalid image file: {}".format(video)
return True
# The code is based on ComfyUI-VideoHelperSuite modification.
class VideoCombine_Adv:
@classmethod
def INPUT_TYPES(s):
#Hide ffmpeg formats if ffmpeg isn't available
if ffmpeg_path is not None:
ffmpeg_formats = ["video/"+x[:-5] for x in folder_paths.get_filename_list("video_formats")]
else:
ffmpeg_formats = []
return {
"required": {
"image_batch": ("IMAGE",),
"frame_rate": (
"INT",
{"default": 8, "min": 1, "step": 1},
),
"loop_count": ("INT", {"default": 0, "min": 0, "max": 100, "step": 1}),
"filename_prefix": ("STRING", {"default": "Comfyui"}),
"format": (["image/gif", "image/webp"] + ffmpeg_formats,),
"pingpong": ("BOOLEAN", {"default": False}),
"save_image": ("BOOLEAN", {"default": True}),
"metadata": ("BOOLEAN", {"default": False}),
},
"hidden": {
"prompt": "PROMPT",
"extra_pnginfo": "EXTRA_PNGINFO",
},
}
RETURN_TYPES = ()
OUTPUT_NODE = True
CATEGORY = "♾️Mixlab/Video"
FUNCTION = "run"
def save_with_tempfile(self, args, metadata, file_path, frames, env):
#Ensure temp directory exists
os.makedirs(folder_paths.get_temp_directory(), exist_ok=True)
metadata_path = os.path.join(folder_paths.get_temp_directory(), "metadata.txt")
#metadata from file should escape = ; # \ and newline
#From my testing, though, only backslashes need escapes and = in particular causes problems
#It is likely better to prioritize future compatibility with containers that don't support
#or shouldn't use the comment tag for embedding metadata
metadata = metadata.replace("\\","\\\\")
metadata = metadata.replace(";","\\;")
metadata = metadata.replace("#","\\#")
#metadata = metadata.replace("=","\\=")
metadata = metadata.replace("\n","\\\n")
with open(metadata_path, "w") as f:
f.write(";FFMETADATA1\n")
f.write(metadata)
args = args[:1] + ["-i", metadata_path] + args[1:] + [file_path]
with subprocess.Popen(args, stdin=subprocess.PIPE, env=env) as proc:
for frame in frames:
proc.stdin.write(frame.tobytes())
def run(
self,
image_batch,
frame_rate: int,
loop_count: int,
filename_prefix="AnimateDiff",
format="image/gif",
pingpong=False,
save_image=True,
metadata=False,
prompt=None,
extra_pnginfo=None,
):
images=image_batch
frames: List[Image.Image] = []
for image in images:
img = 255.0 * image.cpu().numpy()
img = Image.fromarray(np.clip(img, 0, 255).astype(np.uint8))
# resize 保证
# 检查图像的高度是否是2的倍数,如果不是,则调整高度
if img.height % 2 != 0:
img = img.resize((img.width, img.height + 1))
# 检查图像的宽度是否是2的倍数,如果不是,则调整宽度
if img.width % 2 != 0:
img = img.resize((img.width + 1, img.height))
frames.append(img)
# get output information
output_dir = (
folder_paths.get_output_directory()
if save_image
else folder_paths.get_temp_directory()
)
(
full_output_folder,
filename,
counter,
subfolder,
_,
) = folder_paths.get_save_image_path(filename_prefix, output_dir)
metadata = PngInfo()
video_metadata = {}
if prompt is not None:
metadata.add_text("prompt", json.dumps(prompt))
video_metadata["prompt"] = prompt
if extra_pnginfo is not None:
for x in extra_pnginfo:
metadata.add_text(x, json.dumps(extra_pnginfo[x]))
video_metadata[x] = extra_pnginfo[x]
# 取消保存metadata
if metadata==False:
metadata = PngInfo()
# save first frame as png to keep metadata
file = f"{filename}_{counter:05}_.png"
file_path = os.path.join(full_output_folder, file)
frames[0].save(
file_path,
pnginfo=metadata,
compress_level=4,
)
if pingpong:
frames = frames + frames[-2:0:-1]
format_type, format_ext = format.split("/")
file = f"{filename}_{counter:05}_.{format_ext}"
file_path = os.path.join(full_output_folder, file)
if format_type == "image":
# Use pillow directly to save an animated image
frames[0].save(
file_path,
format=format_ext.upper(),
save_all=True,
append_images=frames[1:],
duration=round(1000 / frame_rate),
loop=loop_count,
compress_level=4,
)
else:
# Use ffmpeg to save a video
if ffmpeg_path is None:
#Should never be reachable
raise ProcessLookupError("Could not find ffmpeg")
video_format_path = folder_paths.get_full_path("video_formats", format_ext + ".json")
with open(video_format_path, 'r') as stream:
video_format = json.load(stream)
file = f"{filename}_{counter:05}_.{video_format['extension']}"
file_path = os.path.join(full_output_folder, file)
dimensions = f"{frames[0].width}x{frames[0].height}"
metadata_args = ["-metadata", "comment=" + json.dumps(video_metadata)]
args = [ffmpeg_path, "-v", "error", "-f", "rawvideo", "-pix_fmt", "rgb24",
"-s", dimensions, "-r", str(frame_rate), "-i", "-"] \
+ video_format['main_pass']
# On linux, max arg length is Pagesize * 32 -> 131072
# On windows, this around 32767 but seems to vary wildly by > 500
# in a manor not solely related to other arguments
if os.name == 'posix':
max_arg_length = 4096*32
else:
max_arg_length = 32767 - len(" ".join(args + [metadata_args[0]] + [file_path])) - 1
#test max limit
#metadata_args[1] = metadata_args[1] + "a"*(max_arg_length - len(metadata_args[1])-1)
env=os.environ.copy()
if "environment" in video_format:
env.update(video_format["environment"])
if len(metadata_args[1]) >= max_arg_length:
print(f"Using fallback file for extremely long metadata: {len(metadata_args[1])}/{max_arg_length}")
self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
else:
try:
with subprocess.Popen(args + metadata_args + [file_path],
stdin=subprocess.PIPE, env=env) as proc:
for frame in frames:
proc.stdin.write(frame.tobytes())
except FileNotFoundError as e:
if "winerror" in dir(e) and e.winerror == 206:
print("Metadata was too long. Retrying with fallback file")
self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
else:
raise
except OSError as e:
if "errno" in dir(e) and e.errno == 7:
print("Metadata was too long. Retrying with fallback file")
self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
else:
raise
previews = [
{
"filename": file,
"subfolder": subfolder,
"type": "output" if save_image else "temp",
"format": format,
}
]
return {"ui": {"gifs": previews}}
class VAEEncodeForInpaint_Frames:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"vae": ("VAE", ),
"images": ("IMAGE", ),
"masks": ("MASK", ),
"grow_mask_by": ("INT", {"default": 6, "min": 0, "max": 64, "step": 1}),
}}
FUNCTION = "encode"
RETURN_TYPES = ("LATENT",)
RETURN_NAMES = ("LATENT",)
CATEGORY = "♾️Mixlab/Video"
OUTPUT_NODE = True
INPUT_IS_LIST = True
OUTPUT_IS_LIST = (True,)
def encode(self, vae, images, masks, grow_mask_by=[6]):
vae=vae[0]
grow_mask_by=grow_mask_by[0]
result=[]
for i in range(len(images)):
pixels=images[i]
mask=masks[i]
x = (pixels.shape[1] // 8) * 8
y = (pixels.shape[2] // 8) * 8
mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(pixels.shape[1], pixels.shape[2]), mode="bilinear")
pixels = pixels.clone()
if pixels.shape[1] != x or pixels.shape[2] != y:
x_offset = (pixels.shape[1] % 8) // 2
y_offset = (pixels.shape[2] % 8) // 2
pixels = pixels[:,x_offset:x + x_offset, y_offset:y + y_offset,:]
mask = mask[:,:,x_offset:x + x_offset, y_offset:y + y_offset]
#grow mask by a few pixels to keep things seamless in latent space
if grow_mask_by == 0:
mask_erosion = mask
else:
kernel_tensor = torch.ones((1, 1, grow_mask_by, grow_mask_by))
padding = math.ceil((grow_mask_by - 1) / 2)
mask_erosion = torch.clamp(torch.nn.functional.conv2d(mask.round(), kernel_tensor, padding=padding), 0, 1)
m = (1.0 - mask.round()).squeeze(1)
for i in range(3):
pixels[:,:,:,i] -= 0.5
pixels[:,:,:,i] *= m
pixels[:,:,:,i] += 0.5
t = vae.encode(pixels)
result.append({"samples":t, "noise_mask": (mask_erosion[:,:,:x,:y].round())})
return (result, )
@@ -1,45 +0,0 @@
from comfy.ldm.modules.attention import default, optimized_attention, optimized_attention_masked
from .style_functions import adain, concat_first
class VisualStyleProcessor(object):
def __init__(self,
module_self,
keys_scale: float = 1.0,
enabled: bool = True,
adain_queries: bool = True,
adain_keys: bool = True,
adain_values: bool = False
):
self.module_self = module_self
self.keys_scale = keys_scale
self.enabled = enabled
self.adain_queries = adain_queries
self.adain_keys = adain_keys
self.adain_values = adain_values
def visual_style_forward(self, x, context, value, mask=None):
q = self.module_self.to_q(x)
context = default(context, x)
k = self.module_self.to_k(context)
if value is not None:
v = self.module_self.to_v(value)
del value
else:
v = self.module_self.to_v(context)
if self.enabled:
if self.adain_queries:
q = adain(q)
if self.adain_keys:
k = adain(k)
if self.adain_values:
v = adain(v)
k = concat_first(k, -2, self.keys_scale)
v = concat_first(v, -2)
if mask is None:
out = optimized_attention(q, k, v, self.module_self.heads)
else:
out = optimized_attention_masked(q, k, v, self.module_self.heads, mask)
return self.module_self.to_out(out)
@@ -1,60 +0,0 @@
import torch
from einops import rearrange
from dataclasses import dataclass
T = torch.Tensor
@dataclass(frozen=True)
class StyleAlignedArgs:
share_group_norm: bool = True
share_layer_norm: bool = True,
share_attention: bool = True
adain_queries: bool = True
adain_keys: bool = True
adain_values: bool = False
full_attention_share: bool = False
keys_scale: float = 1.
only_self_level: float = 0.
def expand_first(feat: T, scale=1., ) -> T:
b = feat.shape[0]
feat_style = torch.stack((feat[0], feat[b // 2])).unsqueeze(1)
if scale == 1:
feat_style = feat_style.expand(2, b // 2, *feat.shape[1:])
else:
feat_style = feat_style.repeat(1, b // 2, 1, 1, 1)
feat_style = torch.cat([feat_style[:, :1], scale * feat_style[:, 1:]], dim=1)
return feat_style.reshape(*feat.shape)
def concat_first(feat: T, dim=2, scale=1.) -> T:
feat_style = expand_first(feat, scale=scale)
return torch.cat((feat, feat_style), dim=dim)
def calc_mean_std(feat, eps: float = 1e-5) -> tuple[T, T]:
feat_std = (feat.var(dim=-2, keepdims=True) + eps).sqrt()
feat_mean = feat.mean(dim=-2, keepdims=True)
return feat_mean, feat_std
def adain(feat: T) -> T:
feat_mean, feat_std = calc_mean_std(feat)
feat_style_mean = expand_first(feat_mean)
feat_style_std = expand_first(feat_std)
feat = (feat - feat_mean) / feat_std
feat = feat * feat_style_std + feat_style_mean
return feat
def swapping_attention(key, value, chunk_size=2):
chunk_length = key.size()[0] // chunk_size # [text-condition, null-condition]
reference_image_index = [0] * chunk_length # [0 0 0 0 0]
key = rearrange(key, "(b f) d c -> b f d c", f=chunk_length)
key = key[:, reference_image_index] # ref to all
key = rearrange(key, "b f d c -> (b f) d c")
value = rearrange(value, "(b f) d c -> b f d c", f=chunk_length)
value = value[:, reference_image_index] # ref to all
value = rearrange(value, "b f d c -> (b f) d c")
return key, value
-8
View File
@@ -1,8 +0,0 @@
import folder_paths
# 外挂一个文件,用来编写新的节点
def run(v):
output_dir = folder_paths.get_temp_directory()
print('1323',v,output_dir)
+1 -5
View File
@@ -5,8 +5,4 @@ opencv-python-headless
matplotlib
openai
simple-lama-inpainting
clip-interrogator==0.6.0
transformers>=4.36.0
lark-parser
imageio-ffmpeg
rembg[gpu]
clip-interrogator==0.6.0
+154 -992
View File
File diff suppressed because it is too large Load Diff
+86 -300
View File
@@ -2,46 +2,6 @@ import { app } from '../../../scripts/app.js'
import { $el } from '../../../scripts/ui.js'
import { api } from '../../../scripts/api.js'
//本机安装的插件节点全集
window._nodesAll = null
//获取当前系统的插件,节点清单
function getObjectInfo () {
return new Promise(async (resolve, reject) => {
let url = getUrl()
try {
const response = await fetch(`${url}/object_info`)
const data = await response.json()
resolve(data)
} catch (error) {
reject(error)
}
})
}
const base64Df =
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAAAXNSR0IArs4c6QAAALZJREFUKFOFkLERwjAQBPdbgBkInECGaMLUQDsE0AkRVRAYWqAByxldPPOWHwnw4OBGye1p50UDSoA+W2ABLPN7i+C5dyC6R/uiAUXRQCs0bXoNIu4QPQzAxDKxHoALOrZcqtiyR/T6CXw7+3IGHhkYcy6BOR2izwT8LptG8rbMiCRAUb+CQ6WzQVb0SNOi5Z2/nX35DRyb/ENazhpWKoGwrpD6nICp5c2qogc4of+c7QcrhgF4Aa/aoAFHiL+RAAAAAElFTkSuQmCC'
const parseImageToBase64 = url => {
return new Promise((res, rej) => {
fetch(url)
.then(response => response.blob())
.then(blob => {
const reader = new FileReader()
reader.onloadend = () => {
const base64data = reader.result
res(base64data)
// 在这里可以将base64数据用于进一步处理或显示图片
}
reader.readAsDataURL(blob)
})
.catch(error => {
console.log('发生错误:', error)
})
})
}
function get_position_style (ctx, widget_width, y, node_height) {
const MARGIN = 12 // the margin around the html element
@@ -68,20 +28,20 @@ function get_position_style (ctx, widget_width, y, node_height) {
// height: `${node_height * 0.3 - MARGIN * 2}px`,
// background: '#EEEEEE',
display: 'flex',
flexDirection: 'column',
flexDirection: 'row',
// alignItems: 'center',
justifyContent: 'flex-start'
}
}
async function drawImageToCanvas (imageUrl, sFactor = 320) {
async function drawImageToCanvas (imageUrl) {
var canvas = document.createElement('canvas')
var ctx = canvas.getContext('2d')
var img = new Image()
await new Promise((resolve, reject) => {
img.onload = function () {
var scaleFactor = sFactor / img.width
var scaleFactor = 320 / img.width
var canvasWidth = img.width * scaleFactor
var canvasHeight = img.height * scaleFactor
@@ -106,27 +66,18 @@ async function drawImageToCanvas (imageUrl, sFactor = 320) {
// 可以在这里执行其他操作,比如将Base64数据保存到服务器或显示在页面上
}
async function extractInputAndOutputData (
jsonData,
inputIds = [],
outputIds = []
) {
// workflow
// const workflow=jsonData.workflow;
// const nodes=workflow.nodes;
const data = jsonData.output
let input = []
let output = []
function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
const data = jsonData
const input = []
const output = []
const seed = {}
const seedTitle = {}
for (const id in data) {
if (data.hasOwnProperty(id)) {
let node = app.graph.getNodeById(id)
if (inputIds.includes(id)) {
// let node = app.graph.getNodeById(id)
let options = {}
let options = []
// 模型
try {
if (node.type === 'CheckpointLoaderSimple') {
@@ -159,45 +110,9 @@ async function extractInputAndOutputData (
}
}
if (node.type == 'ImagesPrompt_') {
//图库
// console.log('ImagesPrompt_', data[id])
let image_base64 = data[id].inputs.image_base64
let img_index = 0
let imgsData = JSON.parse(data[id].inputs.upload)
for (let index = 0; index < imgsData.length; index++) {
const imgd = imgsData[index].imgurl
imgsData[index].index = index
//TODO缩放大小
imgsData[index].imgurl = await parseImageToBase64(imgd)
if (image_base64 == imgsData[index].imgurl) {
img_index = index
}
}
options.images = imgsData
delete data[id].inputs.upload
delete data[id].inputs.image_base64
data[id].inputs.imageIndex = img_index
}
if (node.type == 'Color') {
}
if (node.type === 'LoadImage') {
// loadImage的mask支持
let output = node.outputs.filter(ot => ot.type == 'MASK')[0]
if (output.links) {
// 有输出
options.hasMask = true
}
// loadImage的默认图,转为base64
let imgurl = app.graph.getNodeById(id).imgs[0].src
options.defaultImage = await drawImageToCanvas(imgurl, 512)
console.log('#loadImage的默认图', options)
}
input[inputIds.indexOf(id)] = {
...data[id],
title: node.title,
@@ -212,28 +127,18 @@ async function extractInputAndOutputData (
output[outputIds.indexOf(id)] = { ...data[id], title: node.title, id }
}
if (
node.type === 'KSampler' ||
node.type == 'SamplerCustom' ||
node.type === 'ChinesePrompt_Mix' ||
node.type === 'Seed_'
) {
if (node.type === 'KSampler' || node.type == 'SamplerCustom') {
// seed 的类型收集
try {
seed[id] = node.widgets.filter(
w => w.name === 'seed' || w.name == 'noise_seed'
)[0].linkedWidgets[0].value
seedTitle[id] = node.title
} catch (error) {}
}
}
}
// 修复bug,当节点不存在时
input = input.filter(i => i)
output = output.filter(i => i)
return { input, output, seed, seedTitle }
return { input, output, seed }
}
function getUrl () {
@@ -285,10 +190,7 @@ function downloadJsonFile (jsonData, fileName = 'mix_app.json') {
}
async function save (json, download = false, showInfo = true) {
let nodesAll = window._nodesAll || (await getObjectInfo())
console.log('####SAVE', nodesAll, json[0])
console.log('####SAVE', json[0])
const name = json[0],
version = json[5],
share_prefix = json[6], //用于分享的功能扩展
@@ -309,26 +211,12 @@ async function save (json, download = false, showInfo = true) {
try {
let data = await app.graphToPrompt()
//从output数据里把工作流的节点,插件数据统计出来
data.nodesMap = {}
for (const id in data.output) {
data.nodesMap[data.output[id].class_type] =
nodesAll[data.output[id].class_type]
}
let { input, output, seed, seedTitle } = await extractInputAndOutputData(
data,
const { input, output, seed } = extractInputAndOutputData(
data.output,
inputIds,
outputIds
)
let authorAvatar =
localStorage.getItem('_mixlab_author_avatar') || base64Df,
authorName =
localStorage.getItem('_mixlab_author_name') ||
localStorage.getItem('Comfy.userName'),
authorLink = localStorage.getItem('_mixlab_author_link') || ''
data.app = {
name,
description,
@@ -336,16 +224,10 @@ async function save (json, download = false, showInfo = true) {
input,
output,
seed, //控制是fixed 还是random
seedTitle,
share_prefix,
link,
category,
filename: `${name}_${version}.json`,
author: {
avatar: authorAvatar,
name: authorName,
link: authorLink
}
filename: `${name}_${version}.json`
}
try {
@@ -378,13 +260,12 @@ async function save (json, download = false, showInfo = true) {
function getInputsAndOutputs () {
const inputs =
`LoadImage LoadImagesToBatch ImagesPrompt_ VHS_LoadVideo CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
`LoadImage CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
' '
),
outputs =
`SaveTripoSRMesh,PreviewImage,SaveImage,TransparentImage,ShowTextForGPT,VHS_VideoCombine,VideoCombine_Adv,Image Save,SaveImageAndMetadata_,ClipInterrogator`.split(
','
)
outputs = `PreviewImage SaveImage ShowTextForGPT VHS_VideoCombine`.split(
' '
)
let inputsId = [],
outputsId = []
@@ -406,12 +287,7 @@ function getInputsAndOutputs () {
}
app.registerExtension({
name: 'Mixlab.utils.AppInfo',
init () {
if (!window._nodesAll) {
getObjectInfo().then(r => (window._nodesAll = r))
}
},
name: 'Mixlab.app.AppInfo',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'AppInfo') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
@@ -484,164 +360,10 @@ app.registerExtension({
}
})
// author
let author = document.createElement('div')
// author.style=`display: flex`
let authorAvatar = document.createElement('img')
authorAvatar.className = `${'comfy-multiline-input'}`
authorAvatar.style = `outline: none;
border: none;
padding: 4px;
width: 32px;
cursor: pointer;
height: 32px;`
if (localStorage.getItem('_mixlab_author_avatar')) {
authorAvatar.src =
localStorage.getItem('_mixlab_author_avatar') || base64Df
}
let authorAvatarUpload = document.createElement('input')
authorAvatarUpload.type = 'file'
authorAvatarUpload.style = `display:none`
let authorAvatarInput = document.createElement('div')
authorAvatarInput.style = `display: flex;justify-content: flex-start;
align-items: center;`
let authorAvatarInputLabel = document.createElement('p')
authorAvatarInputLabel.innerText = 'Author Avatar'
authorAvatarInputLabel.className = `${'comfy-multiline-input'}`
authorAvatarInputLabel.style = `font-size:12px`
authorAvatar.addEventListener('click', e => {
authorAvatarUpload.click()
})
authorAvatarInputLabel.addEventListener('click', e => {
authorAvatarUpload.click()
})
authorAvatarUpload.addEventListener('change', event => {
const file = event.target.files[0]
const reader = new FileReader()
reader.onload = async e => {
let im = new Image()
im.src = e.target.result
authorAvatar.src = e.target.result
im.onload = () => {
let c = document.createElement('canvas')
let ctx = c.getContext('2d')
c.width = 72
c.height = 72
ctx.drawImage(
im,
0,
0,
im.naturalWidth,
im.naturalHeight,
0,
0,
c.width,
c.height
)
window._mixlab_author_avatar = c.toDataURL()
localStorage.setItem(
'_mixlab_author_avatar',
window._mixlab_author_avatar
)
}
}
// 以文本形式读取文件
reader.readAsDataURL(file)
})
author.appendChild(authorAvatarInput)
authorAvatarInput.appendChild(authorAvatarInputLabel)
authorAvatarInput.appendChild(authorAvatar)
authorAvatarInput.appendChild(authorAvatarUpload)
let authorName = document.createElement('input')
authorName.type = 'text'
authorName.value =
localStorage.getItem('_mixlab_author_name') ||
localStorage.getItem('Comfy.userName')
authorName.placeholder = 'author name'
authorName.className = `${'comfy-multiline-input'}`
authorName.style = `
outline: none;
border: none;
padding: 4px;
width: 100%;
cursor: pointer;
height: 32px;`
let authorNameInput = document.createElement('div')
authorNameInput.style = `display: flex;justify-content: flex-start;
align-items: center;`
let authorNameInputLabel = document.createElement('p')
authorNameInputLabel.innerText = 'Author Name'
authorNameInputLabel.className = `${'comfy-multiline-input'}`
authorNameInputLabel.style = `font-size:12px;width: 110px`
authorName.addEventListener('change', e => {
window._mixlab_author_name = authorName.value.trim()
localStorage.setItem(
'_mixlab_author_name',
window._mixlab_author_name
)
})
author.appendChild(authorNameInput)
authorNameInput.appendChild(authorNameInputLabel)
authorNameInput.appendChild(authorName)
// 社交链接
let authorLink = document.createElement('input')
authorLink.type = 'text'
authorLink.value = localStorage.getItem('_mixlab_author_link') || ''
authorLink.placeholder = 'author link'
authorLink.className = `${'comfy-multiline-input'}`
authorLink.style = `
outline: none;
border: none;
padding: 4px;
width: 100%;
cursor: pointer;
height: 32px;`
let authorLinkInput = document.createElement('div')
authorLinkInput.style = `display: flex;justify-content: flex-start;
align-items: center;`
let authorLinkInputLabel = document.createElement('p')
authorLinkInputLabel.innerText = 'Author Link'
authorLinkInputLabel.className = `${'comfy-multiline-input'}`
authorLinkInputLabel.style = `font-size:12px;width: 110px`
authorLink.addEventListener('change', e => {
window._mixlab_author_link = authorLink.value.trim()
localStorage.setItem(
'_mixlab_author_link',
window._mixlab_author_link
)
})
author.appendChild(authorLinkInput)
authorLinkInput.appendChild(authorLinkInputLabel)
authorLinkInput.appendChild(authorLink)
widget.div.appendChild(author)
let btns = document.createElement('div')
widget.div.appendChild(btns)
btns.appendChild(btn)
btns.appendChild(download)
document.body.appendChild(widget.div)
widget.div.appendChild(btn)
widget.div.appendChild(download)
this.addCustomWidget(widget)
const onRemoved = this.onRemoved
@@ -714,3 +436,67 @@ api.addEventListener('executed', async ({ detail }) => {
}
}
})
app.registerExtension({
name: 'Mixlab.app.AppNode',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'AppNode') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = function () {
orig_nodeCreated?.apply(this, arguments)
console.log('#orig_nodeCreated', this)
let node=this;
const name = this.widgets.filter(w => w.name === 'name')[0]
name.callback = async e => {
let es = e.split('/')
let filename = '',
category = ''
if (es.length == 2) {
category = es[0]
filename = es[1]
} else if (es.length === 1) {
filename = e
}
const res = await api.fetchApi('/mixlab/workflow', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
task: 'my_app',
filename,
category
})
})
if (res.status !== 200) {
throw {
response: await res.json()
}
}
let result=await res.json()
const {app,output:workflow,}=result.data[0].data;
let input=app.input
console.log(input)
if(input.length==1){
let widget = node.inputs.filter(w => w.name === 'input')[0];
let sc=input[0];
LGraphCanvas.prototype._createNodeForInput(
node, //当前node
widget, //当前node里需要自动连线的widget
sc.class_type, //作为input的node type
'IMAGE' // 作为input的node的outputs的name. the input slot type of the target node
)
}
}
this.serialize_widgets = true //需要保存参数
}
}
},
async loadedGraphNode (node, app) {}
})
+2 -8
View File
@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
const repoOwner = 'shadowcz007' // 替换为仓库的所有者
const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
const version = 'v0.22.1'
const version = 'v0.15.0'
fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
.then(response => response.json())
@@ -17,13 +17,7 @@ fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
return
if (latestVersion && latestVersion != version) {
localStorage.setItem('_mixlab_nodes_vesion', latestVersion)
app.ui.dialog.show(`<a style="color: white;
font-size: 18px;
font-weight: 800;
letter-spacing: 2px;
}"
href="https://discord.gg/cXs9vZSqeK">Welcome to Mixlab nodes discord</a>
<h4 style="font-size: 18px;">${repoName} <br>
app.ui.dialog.show(`<h4 style="font-size: 18px;">${repoName} <br>
Latest release version: ${latestVersion}</h4>
<p>Please proceed to the official repository to download the latest version.</p>
<a style="color: #2196F3;
+65 -71
View File
@@ -61,14 +61,14 @@ app.registerExtension({
async getCustomWidgets (app) {
return {
KEY (node, inputName, inputData, app) {
// console.log('##inputData', inputData)
console.log('##inputData', inputData)
const widget = {
type: inputData[0], // the type, CHEESE
name: inputName, // the name, slice
size: [128, 32], // a default size
draw (ctx, node, width, y) {},
computeSize (...args) {
return [128, 32] // a method to compute the current size of the widget
return [128,32] // a method to compute the current size of the widget
},
async serializeValue (nodeId, widgetIndex) {
let data = getLocalData('_mixlab_api_key')
@@ -203,82 +203,76 @@ app.registerExtension({
app.registerExtension({
name: 'Mixlab.GPT.ShowTextForGPT',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeData.name === 'ShowTextForGPT') {
function populate (text) {
text = text.filter(t => t && t?.trim())
if (this.widgets) {
// const pos = this.widgets.findIndex(w => w.name === 'text')
for (let i = 0; i < this.widgets.length; i++) {
if (this.widgets[i].name == 'show_text') this.widgets[i].onRemove?.()
}
this.widgets.length = 1
}
async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name === "ShowTextForGPT") {
function populate(text) {
if (this.widgets) {
const pos = this.widgets.findIndex((w) => w.name === "text");
if (pos !== -1) {
for (let i = pos; i < this.widgets.length; i++) {
this.widgets[i].onRemove?.();
}
this.widgets.length = pos;
}
}
// console.log('ShowTextForGPT',text)
for (let list of text) {
if (list) {
// console.log('#####', list)
const w = ComfyWidgets['STRING'](
this,
'show_text',
['STRING', { multiline: true }],
app
).widget
w.inputEl.readOnly = true
w.inputEl.style.opacity = 0.6
for (let list of text) {
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app).widget;
w.inputEl.readOnly = true;
w.inputEl.style.opacity = 0.6;
try {
if (typeof list != 'string') {
let data = JSON.parse(list)
data = Array.from(data, d => {
return {
...d,
content: decodeURIComponent(d.content)
}
})
list = JSON.stringify(data, null, 2)
try {
let data=JSON.parse(list);
data=Array.from(data,d=>{
return {
...d,
content:decodeURIComponent(d.content)
}
} catch (error) {
console.log(error)
}
w.value = list
})
list=JSON.stringify(data,null,2)
} catch (error) {
// console.log(error)
}
}
w.value =list;
}
// console.log('ShowTextForGPT',this.widgets.length)
requestAnimationFrame(() => {
if (this) {
const sz = this.computeSize()
if (sz[0] < this.size[0]) {
sz[0] = this.size[0]
}
if (sz[1] < this.size[1]) {
sz[1] = this.size[1]
}
this.onResize?.(sz)
app.graph.setDirtyCanvas(true, false)
}
})
}
requestAnimationFrame(() => {
const sz = this.computeSize();
if (sz[0] < this.size[0]) {
sz[0] = this.size[0];
}
if (sz[1] < this.size[1]) {
sz[1] = this.size[1];
}
this.onResize?.(sz);
app.graph.setDirtyCanvas(true, false);
});
}
// When the node is executed we will be sent the input text, display this in the widget
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments)
// console.log('##onExecuted', this, message)
if (message.text) populate.call(this, message.text)
}
// When the node is executed we will be sent the input text, display this in the widget
const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments);
console.log('##',message.text)
populate.call(this, message.text);
};
const onConfigure = nodeType.prototype.onConfigure
nodeType.prototype.onConfigure = function () {
onConfigure?.apply(this, arguments)
if (this.widgets_values?.length) {
populate.call(this, this.widgets_values)
}
}
const onConfigure = nodeType.prototype.onConfigure;
nodeType.prototype.onConfigure = function () {
onConfigure?.apply(this, arguments);
if (this.widgets_values?.length) {
populate.call(this, this.widgets_values);
}
};
this.serialize_widgets = true //需要保存参数
}
}
}
},
})
+5 -404
View File
@@ -1,39 +1,7 @@
import { app } from '../../../scripts/app.js'
import { api } from '../../../scripts/api.js'
// import { ComfyWidgets } from '../../../scripts/widgets.js'
import { ComfyWidgets } from '../../../scripts/widgets.js'
import { $el } from '../../../scripts/ui.js'
import { applyTextReplacements } from '../../../scripts/utils.js'
function loadImageToCanvas (base64Image) {
var img = new Image()
var canvas = document.createElement('canvas')
var ctx = canvas.getContext('2d')
return new Promise((res, rej) => {
img.onload = function () {
// 等比例缩放图片
var width = img.width
var height = img.height
var max_width = 1024
if (width > max_width) {
height *= max_width / width
width = max_width
}
// 设置canvas尺寸
canvas.width = width
canvas.height = height
// 在canvas上绘制图片
ctx.drawImage(img, 0, 0, width, height)
// 将canvas转换为base64图片数据
var canvasData = canvas.toDataURL()
res(canvasData) // canvas转换后的base64图片数据
}
img.src = base64Image
})
}
async function uploadImage (blob, fileType = '.svg', filename) {
// const blob = await (await fetch(src)).blob();
@@ -60,9 +28,6 @@ async function uploadImage (blob, fileType = '.svg', filename) {
return src
}
const base64Df =
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAAAXNSR0IArs4c6QAAALZJREFUKFOFkLERwjAQBPdbgBkInECGaMLUQDsE0AkRVRAYWqAByxldPPOWHwnw4OBGye1p50UDSoA+W2ABLPN7i+C5dyC6R/uiAUXRQCs0bXoNIu4QPQzAxDKxHoALOrZcqtiyR/T6CXw7+3IGHhkYcy6BOR2izwT8LptG8rbMiCRAUb+CQ6WzQVb0SNOi5Z2/nX35DRyb/ENazhpWKoGwrpD6nICp5c2qogc4of+c7QcrhgF4Aa/aoAFHiL+RAAAAAElFTkSuQmCC'
function base64ToBlobFromURL (base64URL, contentType) {
return fetch(base64URL).then(response => response.blob())
}
@@ -143,7 +108,7 @@ function createImage (url) {
})
}
const parseImageToBase64 = url => {
const parseImage = url => {
return new Promise((res, rej) => {
fetch(url)
.then(response => response.blob())
@@ -441,7 +406,9 @@ app.registerExtension({
this.serialize_widgets = true //需要保存参数
}
}
};
},
async loadedGraphNode (node, app) {
// Fires every time a node is constructed
@@ -475,369 +442,3 @@ app.registerExtension({
}
}
})
const createSelect = (imgDiv, select, opts, targetWidget, textWidget) => {
select.style.display = 'block'
let html = ''
let isMatch = false
for (const opt of opts) {
html += `<option value='${opt.keyword}' ${opt.selected ? 'selected' : ''}>${
opt.keyword
}</option>`
if (opt.selected) {
isMatch = true
imgDiv.src = opt.imgurl
// targetWidget.value = opt.keyword
}
}
select.innerHTML = html
if (!isMatch) {
// targetWidget.value = opts[0].keyword
imgDiv.src = opts[0].imgurl
}
// 添加change事件监听器
select.addEventListener('change', async function () {
// 获取选中的选项的值
var selectedOption = select.options[select.selectedIndex].value
let t = opts.filter(opt => opt.keyword === selectedOption)[0]
targetWidget.value = await parseImageToBase64(t.imgurl)
imgDiv.src = targetWidget.value
textWidget.value = t.keyword
})
// console.log(select)
}
app.registerExtension({
name: 'Mixlab.prompt.ImagesPrompt_',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'ImagesPrompt_') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = async function () {
orig_nodeCreated?.apply(this, arguments)
const image_prompt = this.widgets.filter(
w => w.name == 'image_base64'
)[0]
const image_text = this.widgets.filter(w => w.name == 'text')[0]
const node = this
const widget = {
type: 'div',
name: 'upload',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, y, node.size[1])
)
}
}
widget.div = $el('div', {})
// console.log('image_prompt',image_prompt)
const img = new Image()
img.src = image_prompt?.value || base64Df
widget.div.appendChild(img)
const btn = document.createElement('button')
btn.innerText = 'Upload Images JSON'
btn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
`
const select = document.createElement('select')
select.style = `display:none;cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 100px;
`
widget.select = select
// const btn=document.createElement('button');
// btn.innerText='Upload'
btn.addEventListener('click', () => {
let inp = document.createElement('input')
inp.type = 'file'
inp.accept = '.json'
inp.click()
inp.addEventListener('change', event => {
// 获取选择的文件
// [{title,imageUrl}]
const file = event.target.files[0]
this.title = file.name.split('.')[0]
// console.log(file.name.split('.')[0])
// 创建文件读取器
const reader = new FileReader()
// 定义读取完成事件的回调函数
reader.onload = async event => {
// 读取完成后的文本内容
const json = JSON.parse(event.target.result)
console.log(node, json)
widget.value = JSON.stringify(json)
let img = widget.div.querySelector('img')
createSelect(img, select, json, image_prompt, image_text)
image_prompt.value = await parseImageToBase64(json[0].imgurl)
image_text.value = json[0].keyword
if (img) {
img.src = image_prompt.value
}
inp.remove()
}
// 以文本方式读取文件
reader.readAsText(file)
})
})
widget.div.appendChild(btn)
widget.div.appendChild(select)
document.body.appendChild(widget.div)
this.addCustomWidget(widget)
const onRemoved = this.onRemoved
this.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
if (this.onResize) {
this.onResize(this.size)
}
this.serialize_widgets = true //需要保存参数
}
}
},
async loadedGraphNode (node, app) {
if (node.type === 'ImagesPrompt_') {
try {
let prompt = node.widgets.filter(w => w.name === 'image_base64')[0]
let text = node.widgets.filter(w => w.name === 'text')[0]
let uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
// console.log('##prompt',prompt.value)
let img = uploadWidget.div.querySelector('img')
let json = JSON.parse(uploadWidget.value)
for (let index = 0; index < json.length; index++) {
const j = json[index]
let base64 = await parseImageToBase64(j.imgurl)
if (base64 === prompt.value) {
json[index].selected = true
}
}
if (json && json[0]) {
uploadWidget.select.style.display = 'block'
createSelect(img, uploadWidget.select, json, prompt, text)
}
} catch (error) {}
}
}
})
const createInputImageForBatch = (base64, widget) => {
let im = new Image()
im.src = base64
im.style = `width: 88px;`
im.addEventListener('click', e => {
let newValue = []
let items = widget.value?.base64 || []
for (const v of items) {
if (v != base64) newValue.push(v)
}
widget.value.base64 = newValue
im.remove()
})
return im
}
app.registerExtension({
name: 'Mixlab.Comfy.LoadImagesToBatch',
async getCustomWidgets (app) {
return {
IMAGEBASE64 (node, inputName, inputData, app) {
// console.log('##node', node)
const widget = {
value: {
base64: []
}, // 不能[x,x,x]
type: inputData[0], // the type
name: inputName, // the name, slice
size: [128, 32], // a default size
draw (ctx, node, width, y) {},
computeSize (...args) {
return [128, 32] // a method to compute the current size of the widget
}
// serializeValue (nodeId, widgetIndex) {
// return widget.value
// },
}
// widget.something = something; // maybe adds stuff to it
node.addCustomWidget(widget) // adds it to the node
return widget // and returns it.
}
}
},
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'LoadImagesToBatch') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = function () {
orig_nodeCreated?.apply(this, arguments)
let imagesWidget = this.widgets.filter(w => w.name == 'images')[0]
const widget = {
type: 'div',
name: 'image_base64',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, 44, node.size[1])
)
},
serialize: false
}
widget.div = $el('div', {})
document.body.appendChild(widget.div)
let imagePreview = document.createElement('div')
let imagesDiv = document.createElement('div') //显示图片
imagesDiv.className = 'images_preview'
imagesDiv.style = `width: calc(100% - 14px);
display: flex;
flex-wrap: wrap;
padding: 7px; justify-content: space-between;
align-items: center;`
let inputImage = document.createElement('input')
inputImage.type = 'file'
inputImage.style.display = 'none'
inputImage.addEventListener('change', e => {
e.preventDefault()
const file = e.target.files[0]
const reader = new FileReader()
reader.onload = async event => {
let base64 = event.target.result
//压缩图片,控制1024以内
base64 = await loadImageToCanvas(base64)
// console.log(base64)
if (!imagesWidget.value) imagesWidget.value = { base64: [] }
imagesWidget.value.base64.push(base64)
let im = createInputImageForBatch(base64, imagesWidget)
imagesDiv.appendChild(im)
}
reader.readAsDataURL(file)
})
const btn = document.createElement('button')
btn.innerText = 'Upload Image'
btn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
`
btn.addEventListener('click', e => {
e.preventDefault()
inputImage.click()
})
widget.div.appendChild(imagePreview)
imagePreview.appendChild(imagesDiv)
imagePreview.appendChild(btn)
imagePreview.appendChild(inputImage)
this.addCustomWidget(widget)
// document.addEventListener('wheel', handleMouseWheel)
const onRemoved = this.onRemoved
this.onRemoved = () => {
inputImage.remove()
widget.div.remove()
try {
// document.removeEventListener('wheel', handleMouseWheel)
} catch (error) {
console.log(error)
}
return onRemoved?.()
}
this.serialize_widgets = true //需要保存参数
}
}
if (nodeData.name === 'SaveImageAndMetadata_') {
const onNodeCreated = nodeType.prototype.onNodeCreated
// /web/extensions/core/saveImageExtraOutput.js
nodeType.prototype.onNodeCreated = function () {
const r = onNodeCreated
? onNodeCreated.apply(this, arguments)
: undefined
const widget = this.widgets.find(w => w.name === 'filename_prefix')
widget.serializeValue = () => {
return applyTextReplacements(app, widget.value)
}
return r
}
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments)
console.log('##onExecuted', this, message)
//TODO 是否 保存base64
if (message.base64) {
if (Array.isArray(message.base64)) {
}
}
}
}
},
async loadedGraphNode (node, app) {
if (node.type === 'LoadImagesToBatch') {
// await sleep(0)
let imagesWidget = node.widgets.filter(w => w.name === 'images')[0]
let imagePreview = node.widgets.filter(w => w.name == 'image_base64')[0]
let pre = imagePreview.div.querySelector('.images_preview')
for (const d of imagesWidget.value?.base64 || []) {
let im = createInputImageForBatch(d, imagesWidget)
pre.appendChild(im)
}
}
}
})
+14 -566
View File
@@ -1,70 +1,8 @@
import { app } from '../../../scripts/app.js'
import { api } from '../../../scripts/api.js'
// import { api } from '../../../scripts/api.js'
import { ComfyWidgets } from '../../../scripts/widgets.js'
import { $el } from '../../../scripts/ui.js'
function downloadJsonFile (jsonData, fileName = 'grid.json') {
const dataString = JSON.stringify(jsonData)
const blob = new Blob([dataString], { type: 'application/json' })
const url = URL.createObjectURL(blob)
const link = document.createElement('a')
link.href = url
link.download = fileName
link.click()
// 释放URL对象
setTimeout(() => {
URL.revokeObjectURL(url)
}, 0)
}
function createSelectWithOptions (options) {
const select = document.createElement('select')
options.forEach(option => {
const optionElement = document.createElement('option')
optionElement.text = option
optionElement.value = option
select.appendChild(optionElement)
})
select.style = `cursor: pointer;
font-weight: 300;
height: 30px;
min-width: 122px;
position: absolute;
top: 24px;
left: 88px;
z-index: 999999999999999;
`
return select
}
function drawCanvasWithText (w, h, tag, color = 'rgba(255,255,255,0.4)') {
const canvas = document.createElement('canvas')
const ctx = canvas.getContext('2d')
// 设置画布大小
canvas.width = w
canvas.height = h
// 绘制白色背景
ctx.fillStyle = color
ctx.fillRect(0, 0, canvas.width, canvas.height)
// 绘制文字
ctx.fillStyle = '#000000'
ctx.font = '20px Arial'
ctx.fillText(tag, 50, 50)
// 导出为Base64
const base64 = canvas.toDataURL()
return base64
}
function get_position_style (ctx, widget_width, y, node_height) {
const MARGIN = 4 // the margin around the html element
@@ -218,29 +156,32 @@ const parseSvg = async svgContent => {
return { data, image: base64, svgElement }
}
function findImages (nodeId) {
function findImages(nodeId) {
// 检查当前节点是否有 imgs 字段
const n = app.graph.getNodeById(nodeId)
if (n.imgs) {
return n.imgs
return n.imgs;
}
// 检查当前节点的 inputs 是否有 image 字段
if (n.inputs) {
for (let i = 0; i < n.inputs.length; i++) {
if (n.inputs[i].name === 'image' || n.inputs[i].name === 'images') {
if (n.inputs[i].name==='image'||n.inputs[i].name==='images') {
// 获取新的 nodeId,并递归调用 findImages 函数
var linkId = n.inputs[i]?.link
var linkId = n.inputs[i]?.link;
var origin_id = app.graph.links[linkId].origin_id
return findImages(origin_id)
return findImages(origin_id);
}
}
}
// 如果没有找到 imgs 字段或者 image 字段,则返回 null
return null
return null;
}
async function setArea (cw, ch, topBase64, base64, data, fn) {
let displayHeight = Math.round(window.screen.availHeight * 0.8)
let div = document.createElement('div')
@@ -412,196 +353,6 @@ async function setArea (cw, ch, topBase64, base64, data, fn) {
}
}
async function setAreaTags (cw, ch, grids, fn) {
let base64 = drawCanvasWithText(cw, ch, '', 'white')
let displayHeight = Math.round(window.screen.availHeight * 0.8)
let div = document.createElement('div')
div.innerHTML = `
<div id='ml_overlay' style='position: absolute;top:0;background: #251f1fc4;
height: 100vh;
z-index:999999;
width: 100%;'>
<img id='ml_video' style='position: absolute;
height: ${displayHeight}px;user-select: none;
-webkit-user-drag: none;
outline: 2px solid #eaeaea;
box-shadow: 8px 9px 17px #575757;' />
${Array.from(grids, g => {
const { label: tag, grid } = g
const [dx, dy, dw, dh] = grid
const base64Data = drawCanvasWithText(dw, dh, tag)
let x = 0,
y = 0,
width = (cw * displayHeight) / ch,
height = displayHeight
let imgWidth = cw
let imgHeight = ch
if (dw > 0 && dh > 0) {
// 相同尺寸窗口,恢复选区
x = (width * dx) / imgWidth
y = (height * dy) / imgHeight
width = (width * dw) / imgWidth
height = (height * dh) / imgHeight
}
return `<div class='ml_selection'
data-tag="${tag}"
style='position:absolute;
border: 2px dashed red;
pointer-events: none;
background-image: url("${base64Data}");
background-repeat: no-repeat;
background-size: cover;
left:${x}px;
top:${y}px;
width:${width}px;
height:${height}px;
'></div>`
})}
<div class="mx_close"> X </div>
</div>`
// document.body.querySelector('#ml_overlay')
document.body.appendChild(div)
const tags = Array.from(grids, g => g.label)
let select = createSelectWithOptions(tags)
document.body.appendChild(select)
let img = div.querySelector('#ml_video')
// let overlay = div.querySelector('#ml_overlay')
let selections = [...div.querySelectorAll('.ml_selection')]
let selection = selections.filter(
s => s.getAttribute('data-tag') === select.value
)[0]
select.addEventListener('change', e => {
selection = selections.filter(
s => s.getAttribute('data-tag') === select.value
)[0]
})
// console.log(select.value,selection)
let close = div.querySelector('.mx_close')
let startX, startY, endX, endY
let start = false
let setDone = false
// Set video source
img.src = base64
// canvas.toDataURL();
close.style = `cursor: pointer;
position: fixed;
left: 12px;
top: 12px;
z-index: 99999999;
background: black;
width: 44px;
height: 44px;
text-align: center;
line-height: 44px;`
// Add mouse events
img.addEventListener('mousedown', startSelection)
img.addEventListener('mousemove', updateSelection)
img.addEventListener('mouseup', endSelection)
const removeDiv = () => {
div.remove()
select?.remove()
close.removeEventListener('click', removeDiv)
img.removeEventListener('mousedown', startSelection)
img.removeEventListener('mousemove', updateSelection)
img.removeEventListener('mouseup', endSelection)
img.removeEventListener('mousedown', setDoneCheck)
}
close.addEventListener('click', removeDiv)
const setDoneCheck = event => {
console.log(setDone)
if (setDone) {
img.addEventListener('mousedown', startSelection)
img.addEventListener('mousemove', updateSelection)
img.addEventListener('mouseup', endSelection)
setDone = false
start = false
startX = event.clientX
startY = event.clientY
}
}
img.addEventListener('mousedown', setDoneCheck)
function remove () {
img.removeEventListener('mousedown', startSelection)
img.removeEventListener('mousemove', updateSelection)
img.removeEventListener('mouseup', endSelection)
setDone = true
// select?.remove()
}
function startSelection (event) {
if (start == false) {
startX = event.clientX
startY = event.clientY
updateSelection(event)
start = true
} else {
}
}
function updateSelection (event) {
endX = event.clientX
endY = event.clientY
// Calculate width, height, and coordinates
let width = Math.abs(endX - startX)
let height = Math.abs(endY - startY)
let left = Math.min(startX, endX)
let top = Math.min(startY, endY)
// Set selection style
selection.style.left = left + 'px'
selection.style.top = top + 'px'
selection.style.width = width + 'px'
selection.style.height = height + 'px'
}
function endSelection (event) {
endX = event.clientX
endY = event.clientY
// 获取img元素的真实宽度和高度
let imgWidth = img.naturalWidth
let imgHeight = img.naturalHeight
// 换算起始坐标
let realStartX = (startX / img.offsetWidth) * imgWidth
let realStartY = (startY / img.offsetHeight) * imgHeight
// 换算起始坐标
let realEndX = (endX / img.offsetWidth) * imgWidth
let realEndY = (endY / img.offsetHeight) * imgHeight
startX = realStartX
startY = realStartY
endX = realEndX
endY = realEndY
// Calculate width, height, and coordinates
let width = Math.round(Math.abs(endX - startX))
let height = Math.round(Math.abs(endY - startY))
let left = Math.round(Math.min(startX, endX))
let top = Math.round(Math.min(startY, endY))
if (width <= 0 && height <= 0) return remove()
if (!!fn) fn(select.value, left, top, width, height)
remove()
}
}
app.registerExtension({
name: 'Mixlab.layer.ShowLayer',
async getCustomWidgets (app) {
@@ -847,8 +598,8 @@ app.registerExtension({
}
try {
console.log('this.inputs', this.id)
let imgs = findImages(this.id)
let imgs=findImages(this.id)
// let topLinkId = this.inputs[0].link
// let topNodeId = app.graph.links[topLinkId].origin_id
let topIm = imgs[0]
@@ -856,9 +607,9 @@ app.registerExtension({
let linkId = this.inputs[3].link
let nodeId = app.graph.links[linkId].origin_id
// console.log(linkId,this.inputs)
let imgs2 = findImages(nodeId)
let imgs2=findImages(nodeId)
let im = imgs2[0]
console.log(topIm, im)
console.log(topIm,im)
// let src = im.src
setArea(
im.naturalWidth,
@@ -890,306 +641,3 @@ app.registerExtension({
}
}
})
app.registerExtension({
name: 'Mixlab.layer.GridInput',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'GridInput') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = async function () {
orig_nodeCreated?.apply(this, arguments)
const grids_widget = this.widgets.filter(w => w.name == 'grids')[0]
const widget = {
type: 'div',
name: 'upload',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, y, node.size[1]),
{
justifyContent: 'flex-start'
}
)
}
}
widget.div = $el('div', {})
const addBtn = document.createElement('button')
addBtn.innerText = 'Add Box'
addBtn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
`
const vbtn = document.createElement('button')
vbtn.innerText = 'Set Box'
vbtn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
`
const btn = document.createElement('button')
btn.innerText = 'Upload JSON'
btn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
`
addBtn.addEventListener('click', () => {
const { width, height, grids } = JSON.parse(grids_widget.value)
grids.push({
label: 'background',
grid: [12, 12, width - 24, height - 24]
})
grids_widget.value = JSON.stringify(
{
width,
height,
grids
},
null,
2
)
})
vbtn.addEventListener('click', () => {
const { width, height, grids } = JSON.parse(grids_widget.value)
setAreaTags(width, height, grids, (tag, x, y, w, h) => {
grids_widget.value = JSON.stringify(
{
width,
height,
grids: Array.from(grids, g => {
if (g.label === tag) {
g.grid = [x, y, w, h]
}
return g
})
},
null,
2
)
})
})
btn.addEventListener('click', () => {
let inp = document.createElement('input')
inp.type = 'file'
inp.accept = '.json'
inp.click()
inp.addEventListener('change', event => {
// 获取选择的文件
const file = event.target.files[0]
this.title = file.name.split('.')[0]
// console.log(file.name.split('.')[0])
// 创建文件读取器
const reader = new FileReader()
// 定义读取完成事件的回调函数
reader.onload = event => {
// 读取完成后的文本内容
const fileContent = JSON.parse(event.target.result)
const grids = fileContent
grids_widget.value = JSON.stringify(grids, null, 2)
// widget.value = grids
inp.remove()
}
// 以文本方式读取文件
reader.readAsText(file)
})
})
widget.div.appendChild(addBtn)
widget.div.appendChild(vbtn)
widget.div.appendChild(btn)
document.body.appendChild(widget.div)
this.addCustomWidget(widget)
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
const r = onExecuted?.apply?.(this, arguments)
let json = message.json
if (json) {
json = {
width: json[0],
height: json[1],
grids: json[2]
}
grids_widget.value = JSON.stringify(json, null, 2)
// widget.value = json
}
return r
}
const onRemoved = this.onRemoved
this.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
if (this.onResize) {
this.onResize(this.size)
}
this.serialize_widgets = true //需要保存参数
}
}
},
async loadedGraphNode (node, app) {
if (node.type === 'GridInput') {
try {
const grids_widget = node.widgets.filter(w => w.name == 'grids')[0]
const { width, height, grids } = JSON.parse(grids_widget.value)
console.log('#GridInput', node, grids)
const div = node.widgets.filter(w => w.name == 'upload')[0]
div.div.querySelector('select').innerHTML = Array.from(
grids,
g => `<option value="${g.label}">${g.label}</option>`
).join('')
} catch (error) {}
}
}
})
app.registerExtension({
name: 'Mixlab.layer.GridDisplayAndSave',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeType.comfyClass == 'GridDisplayAndSave') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = async function () {
orig_nodeCreated?.apply(this, arguments)
const grids_widget = this.widgets.filter(w => w.name == 'grids')[0]
console.log('GridDisplayAndSave', grids_widget)
const widget = {
type: 'div',
name: 'save_json',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, y, node.size[1]),
{
justifyContent: 'flex-start',
flexDirection: 'column'
}
)
}
}
widget.div = $el('div', {})
const btn = document.createElement('button')
btn.innerText = 'Save JSON'
btn.style = `cursor: pointer;
font-weight: 300;
margin: 2px;
color: var(--descrip-text);
background-color: var(--comfy-input-bg);
border-radius: 8px;
border-color: var(--border-color);
border-style: solid;height: 30px;min-width: 122px;
max-width: 122px;
`
btn.addEventListener('click', () => {
if (window._mixlab_grid)
downloadJsonFile(
window._mixlab_grid,
this.widgets.filter(w => w.name == 'filename_prefix')[0]?.value +
'_grid.json'
)
})
widget.div.appendChild(btn)
document.body.appendChild(widget.div)
this.addCustomWidget(widget)
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
const r = onExecuted?.apply?.(this, arguments)
let save_json = this.widgets.filter(d => d.name == 'save_json')[0]
let div = save_json?.div
// console.log('Test',message)
let image = message.image[0]
let json = message.json
if (image) {
const { filename, subfolder, type } = image
if (!div.querySelector('img')) {
let im = new Image()
div.appendChild(im)
im.style.width = '100%'
}
div.querySelector('img').src = api.apiURL(
`/view?filename=${encodeURIComponent(
filename
)}&type=${type}&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
)
window._mixlab_grid = {
width: json[0],
height: json[1],
grids: json[2]
}
// console.log(src)
}
this.onResize?.(this.size)
return r
}
const onRemoved = this.onRemoved
this.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
if (this.onResize) {
this.onResize(this.size)
}
this.serialize_widgets = true //需要保存参数
}
}
},
async loadedGraphNode (node, app) {
if (node.type === 'GridDisplayAndSave') {
try {
let grids_widget = node.widgets.filter(w => w.name === 'grids')[0]
// let ks = getLocalData(`_mixlab_PromptSlide`)
let uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
// console.log('##widget', uploadWidget.value)
let grids = JSON.parse(uploadWidget.value)
} catch (error) {}
}
}
})
+3 -2
View File
@@ -408,7 +408,7 @@ const _createResult = async (node, widget, message) => {
const width = node.size[0] * 0.5 - 12
let height_add = 0
for (let index = 0; index < message._images.length; index++) {
const imgs = message._images[index]
@@ -559,7 +559,8 @@ app.registerExtension({
let cards = widget.div.querySelectorAll('.card')
if (cards.length == 0) node.size = [280, 120]
if(widget.value) _createResult(node, widget, widget.value)
_createResult(node, widget, widget.value)
}
}
})
+1 -28
View File
@@ -46,18 +46,6 @@ const smart_connect_config_input = [
node_widget_name: 'image',
inputNodeName: 'LoadImage',
inputNode_output_name: 'IMAGE'
},
{
node_type: 'TripoSRSampler_',
node_widget_name: 'image',
inputNodeName: 'LoadImagesToBatch',
inputNode_output_name: 'IMAGE'
},
{
node_type: 'TripoSRSampler_',
node_widget_name: 'mask',
inputNodeName: 'RembgNode_Mix',
inputNode_output_name: 'masks'
}
]
@@ -86,18 +74,6 @@ const smart_connect_config_output = [
outputNodeName: 'SaveImage',
outputNode_input_name: 'images'
},
{
node_type: 'VAEDecode',
node_output_name: 'IMAGE',
outputNodeName: 'AppInfo',
outputNode_input_name: 'IMAGE'
},
{
node_type: 'VAEDecode',
node_output_name: 'IMAGE',
outputNodeName: 'SaveImageAndMetadata_',
outputNode_input_name: 'images'
},
{
node_type: 'Moondream',
node_output_name: 'STRING',
@@ -205,10 +181,7 @@ export function smart_init () {
]
let node_slotType = config[0]
// 如果input没有,则创建
if (
!node.inputs?.filter(inp => inp.name === widget.name)[0] ||
!node.inputs
)
if (!node.inputs?.filter(inp => inp.name === widget.name)[0]||!node.inputs)
convertToInput(node, widget, config)
input_node.connectByType(inputNode_slot, node, node_slotType)
}
+41 -533
View File
@@ -1,6 +1,5 @@
import { app } from '../../../scripts/app.js'
import { closeIcon } from './svg_icons.js'
import { api } from '../../../scripts/api.js'
import {
GroupNodeConfig,
@@ -9,172 +8,6 @@ import {
import { smart_init, addSmartMenu } from './smart_connect.js'
let isScriptLoaded = {}
function loadExternalScript (url) {
return new Promise((resolve, reject) => {
if (isScriptLoaded[url]) {
resolve()
return
}
const script = document.createElement('script')
script.src = url
script.onload = () => {
isScriptLoaded[url] = true
resolve()
}
script.onerror = reject
document.head.appendChild(script)
})
}
//
function createChart (chartDom, nodes) {
var myChart = echarts.init(chartDom)
var option
console.log(nodes)
option = {
series: [
{
type: 'treemap',
data: [
{
name: 'nodeA',
value: 10,
children: Array.from(nodes, n => {
return {
name: n.type,
value: n.count
}
})
}
]
}
]
}
option && myChart.setOption(option)
}
async function createNodesCharts () {
await loadExternalScript(
'/extensions/comfyui-mixlab-nodes/lib/echarts.min.js'
)
const templates = await loadTemplate()
var nodes = {}
Array.from(templates, t => {
let j = JSON.parse(t.data)
for (let node of j.nodes) {
if (!nodes[node.type]) nodes[node.type] = { type: node.type, count: 0 }
nodes[node.type].count++
}
})
nodes = Object.values(nodes).sort((a, b) => b.count - a.count)
const menu = document.querySelector('.comfy-menu')
const separator = document.createElement('div')
separator.style = `margin: 20px 0px;
width: 100%;
height: 1px;
background: var(--border-color);
`
menu.append(separator)
const appsButton = document.createElement('button')
appsButton.textContent = 'Nodes'
appsButton.onclick = () => {
let div = document.querySelector('#mixlab_apps')
if (!div) {
div = document.createElement('div')
div.id = 'mixlab_apps'
document.body.appendChild(div)
let btn = document.createElement('div')
btn.style = `display: flex;
width: calc(100% - 24px);
justify-content: space-between;
align-items: center;
padding: 0 12px;
height: 44px;`
let btnB = document.createElement('button')
let textB = document.createElement('p')
btn.appendChild(textB)
btn.appendChild(btnB)
textB.style.fontSize = '12px'
textB.innerText = `Nodes`
btnB.style = `float: right; border: none; color: var(--input-text);
background-color: var(--comfy-input-bg); border-color: var(--border-color);cursor: pointer;`
btnB.addEventListener('click', () => {
div.style.display = 'none'
})
btnB.innerText = 'X'
// 悬浮框拖动事件
div.addEventListener('mousedown', function (e) {
var startX = e.clientX
var startY = e.clientY
var offsetX = div.offsetLeft
var offsetY = div.offsetTop
function moveBox (e) {
var newX = e.clientX
var newY = e.clientY
var deltaX = newX - startX
var deltaY = newY - startY
div.style.left = offsetX + deltaX + 'px'
div.style.top = offsetY + deltaY + 'px'
localStorage.setItem(
'mixlab_app_pannel',
JSON.stringify({ x: div.style.left, y: div.style.top })
)
}
function stopMoving () {
document.removeEventListener('mousemove', moveBox)
document.removeEventListener('mouseup', stopMoving)
}
document.addEventListener('mousemove', moveBox)
document.addEventListener('mouseup', stopMoving)
})
div.appendChild(btn)
let chartDom = document.createElement('div')
chartDom.style = `height:80vh;width:450px`
chartDom.className = 'chart'
div.appendChild(chartDom)
}
if (div.style.display == 'flex') {
div.style.display = 'none'
} else {
let pos = JSON.parse(
localStorage.getItem('mixlab_app_pannel') ||
JSON.stringify({ x: 0, y: 0 })
)
div.style = `
flex-direction: column;
align-items: end;
display:flex;
position: absolute;
top: ${pos.y}; left: ${pos.x}; width: 450px;
color: var(--descrip-text);
background-color: var(--comfy-menu-bg);
padding: 10px;
border: 1px solid black;z-index: 999999999;padding-top: 0;`
}
createChart(div.querySelector('.chart'), nodes)
}
menu.append(appsButton)
}
function copyNodeValues (src, dest) {
// title
dest.title = src.title
@@ -282,33 +115,28 @@ function get_url () {
async function get_my_app (filename = null, category = '') {
let url = get_url()
let data = null
try {
const res = await fetch(`${url}/mixlab/workflow`, {
method: 'POST',
body: JSON.stringify({
task: 'my_app',
filename,
category,
admin: true
})
const res = await fetch(`${url}/mixlab/workflow`, {
method: 'POST',
body: JSON.stringify({
task: 'my_app',
filename,
category,
admin: true
})
let result = await res.json()
data = []
})
let result = await res.json()
let data = []
try {
for (const res of result.data) {
let { app, workflow } = res.data;
if (app?.filename) data.push({
let { app, workflow } = res.data
if (app.filename)
data.push({
...app,
data: workflow,
date: res.date
})
}
} catch (error) {
console.log(error)
}
} catch (error) {}
return data
}
@@ -363,14 +191,7 @@ injectCSS(`::-webkit-scrollbar {
animation-name: loading_mixlab;
animation-duration: 2s;
animation-iteration-count: infinite;
}
.dynamic_prompt{
border-left: 2px solid var(--input-text);
}
`)
}`)
async function getCustomnodeMappings (mode = 'url') {
// mode = "local";
@@ -795,106 +616,6 @@ function createModal (url, markdown, title) {
div.appendChild(bgElement)
}
const loadTemplate = async () => {
const id = 'Comfy.NodeTemplates'
const file = 'comfy.templates.json'
let templates = []
if (app.storageLocation === 'server') {
if (app.isNewUserSession) {
// New user so migrate existing templates
const json = localStorage.getItem(id)
if (json) {
templates = JSON.parse(json)
}
await api.storeUserData(file, json, { stringify: false })
} else {
const res = await api.getUserData(file)
if (res.status === 200) {
try {
templates = await res.json()
} catch (error) {}
} else if (res.status !== 404) {
console.error(res.status + ' ' + res.statusText)
}
}
} else {
const json = localStorage.getItem(id)
if (json) {
templates = JSON.parse(json)
}
}
return templates ?? []
}
function drawBadge (node, orig, restArgs) {
let ctx = restArgs[0]
const r = orig?.apply?.(node, restArgs)
if (
!node.flags.collapsed &&
node.constructor.title_mode != LiteGraph.NO_TITLE
) {
let text = `#${node.id} `
let nick = node.getNickname()
if (nick) {
if (nick == 'ComfyUI') {
nick = '🦊'
}
if (nick.length > 25) {
text += nick.substring(0, 23) + '..'
} else {
text += nick
}
}
if (text != '') {
let fgColor = 'white'
let bgColor = '#0F1F0F'
let visible = true
ctx.save()
ctx.font = '12px sans-serif'
const sz = ctx.measureText(text)
ctx.fillStyle = bgColor
ctx.beginPath()
ctx.roundRect(
node.size[0] - sz.width - 12,
-LiteGraph.NODE_TITLE_HEIGHT - 20,
sz.width + 12,
20,
5
)
ctx.fill()
ctx.fillStyle = fgColor
ctx.fillText(
text,
node.size[0] - sz.width - 6,
-LiteGraph.NODE_TITLE_HEIGHT - 6
)
ctx.restore()
if (node.has_errors) {
ctx.save()
ctx.font = 'bold 14px sans-serif'
const sz2 = ctx.measureText(node.type)
ctx.fillStyle = 'white'
ctx.fillText(
node.type,
node.size[0] / 2 - sz2.width / 2,
node.size[1] / 2
)
ctx.restore()
}
}
}
return r
}
app.registerExtension({
name: 'Comfy.Mixlab.ui',
init () {
@@ -944,21 +665,7 @@ app.registerExtension({
}
]
if (node.widgets) {
// let text_widget = node.widgets.filter(
// w => w.name === 'text' && typeof w.value == 'string'
// )
// if (text_widget && text_widget.length == 1) {
// opts.push({
// content: 'Text-to-Text ♾️Mixlab', // with a name
// callback: () => {
// LGraphCanvas.prototype.text2text(node)
// } // and the callback
// })
// }
}
opts = addSmartMenu(opts, node)
opts = addSmartMenu(opts,node)
// if (node.type == 'CLIPTextEncode') {
// // 则出现 randomPrompt
@@ -999,6 +706,16 @@ app.registerExtension({
const options = getGroupMenuOptions.apply(this, arguments) // start by calling the stored one
node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
// templete
const key = 'Comfy.NodeTemplates'
let templates = localStorage.getItem(key)
if (templates) {
templates = JSON.parse(templates)
} else {
templates = []
}
const store = () => localStorage.setItem(key, JSON.stringify(templates))
return [
{
content: 'Clone Group ♾️Mixlab', // with a name
@@ -1056,7 +773,7 @@ app.registerExtension({
localStorage.setItem('litegrapheditor_clipboard', old)
}
clipboardAction(async () => {
clipboardAction(() => {
let name = group.title + ' ♾️Mixlab'
let nodes = group._nodes
@@ -1069,8 +786,6 @@ app.registerExtension({
const nodeData = node.serialize()
let groupData = GroupNodeHandler.getGroupData(node)
// console.log('groupData',GroupNodeHandler.isGroupNode(node),groupData)
if (groupData) {
groupData = groupData.nodeData
if (!data.groupNodes) {
@@ -1081,32 +796,11 @@ app.registerExtension({
}
}
// templete
const store = async nt => {
const id = 'Comfy.NodeTemplates'
const file = 'comfy.templates.json'
let templates = await loadTemplate()
templates.push(nt)
if (app.storageLocation === 'server') {
const ts = JSON.stringify(templates, undefined, 4)
localStorage.setItem(id, ts) // Backwards compatibility
try {
await api.storeUserData(file, ts, {
stringify: false
})
} catch (error) {
console.error(error)
alert(error.message)
}
} else {
localStorage.setItem(id, JSON.stringify(templates))
}
}
console.log('data', data)
store({
templates.push({
name,
data: JSON.stringify(data)
})
store()
})
} // and the callback
},
@@ -1138,57 +832,6 @@ app.registerExtension({
(this.canvas.height * 0.5) / (this.ds.scale * dpr) // 考虑设备像素比
this.setDirty(true, true)
}
// 支持app模式的json
const loadAppJson = async data => {
let workflow
try {
let w = JSON.parse(data)
if (w.app && w.output) workflow = w.workflow
} catch (err) {}
if (workflow && workflow.version && workflow.nodes && workflow.extra) {
await app.loadGraphData(workflow)
}
}
if (!window._mixlab_app_paste_listener) {
window._mixlab_app_paste_listener = true
//粘贴json的事件
document.addEventListener('paste', async e => {
// ctrl+shift+v is used to paste nodes with connections
// this is handled by litegraph
if (this.shiftDown) return
let data = e.clipboardData || window.clipboardData
// No image found. Look for node data
data = data.getData('text/plain')
loadAppJson(data)
})
// 把json往里 拖
document.addEventListener('drop', async event => {
event.preventDefault()
event.stopPropagation()
// Dragging from Chrome->Firefox there is a file but its a bmp, so ignore that
if (
event.dataTransfer.files.length &&
event.dataTransfer.files[0].type == 'application/json'
) {
const reader = new FileReader()
reader.onload = async () => {
loadAppJson(reader.result)
}
reader.readAsText(event.dataTransfer.files[0])
}
})
}
},
setup () {
setTimeout(async () => {
@@ -1196,9 +839,6 @@ app.registerExtension({
const orig = LGraphCanvas.prototype.getCanvasMenuOptions
const apps = await get_my_app()
if (!apps) return
console.log('apps',apps)
let apps_map = { 0: [] }
@@ -1213,7 +853,7 @@ app.registerExtension({
let apps_opts = []
for (const category in apps_map) {
// console.log('category', typeof category)
console.log('category', typeof category)
if (category === '0') {
apps_opts.push(
...Array.from(apps_map[category], a => {
@@ -1223,30 +863,8 @@ app.registerExtension({
has_submenu: false,
callback: async () => {
try {
let ddd = await get_my_app(a.filename)
if (!ddd) return
let item = ddd[0]
let item = (await get_my_app(a.filename))[0]
if (item) {
if (item.author) {
// 有作者信息
if (item.author.avatar)
localStorage.setItem(
'_mixlab_author_avatar',
item.author.avatar
)
if (item.author.name)
localStorage.setItem(
'_mixlab_author_name',
item.author.name
)
if (item.author.link)
localStorage.setItem(
'_mixlab_author_link',
item.author.link
)
}
// console.log(item.data)
app.loadGraphData(item.data)
setTimeout(() => {
@@ -1274,31 +892,8 @@ app.registerExtension({
content: `${a.name}_${a.version}`,
callback: async () => {
try {
let ddd = await get_my_app(a.filename, a.category)
if (!ddd) return
let item = ddd[0]
let item = (await get_my_app(a.filename, a.category))[0]
if (item) {
console.log(item)
if (item.author) {
// 有作者信息
if (item.author.avatar)
localStorage.setItem(
'_mixlab_author_avatar',
item.author.avatar
)
if (item.author.name)
localStorage.setItem(
'_mixlab_author_name',
item.author.name
)
if (item.author.link)
localStorage.setItem(
'_mixlab_author_link',
item.author.link
)
}
// console.log(item.data)
app.loadGraphData(item.data)
setTimeout(() => {
@@ -1565,68 +1160,19 @@ app.registerExtension({
document.body.appendChild(div)
}
},
apps_opts.length > 0
? {
content: 'Workflow App ♾️Mixlab',
has_submenu: true,
disabled: false,
submenu: {
options: apps_opts
}
}
: null
{
content: 'Workflow App ♾️Mixlab',
has_submenu: true,
disabled: false,
submenu: {
options: apps_opts
}
}
)
return options
}
}, 1000)
// createNodesCharts()
},
nodeCreated (node) {
if (node.widgets) {
// Locate dynamic prompt text widgets
// Include any widgets with dynamicPrompts set to true, and customtext
for (let index = 0; index < node.widgets.length; index++) {
const widget = node.widgets[index]
if (
(widget.type === 'customtext' && widget.dynamicPrompts !== false) ||
widget.dynamicPrompts
) {
widget.element.classList.add('dynamic_prompt')
widget.element.addEventListener('mouseover', e => {
// console.log(node.widgets_values[index])
if (node.widgets_values && node.widgets_values[index])
widget.element.setAttribute('title', node.widgets_values[index])
})
}
}
}
fetch('manager/badge_mode').then(r => {
if (r.status === 404) {
// 右上角的badge是否已经绘制
if (!node.badge_enabled) {
if (!node.getNickname) {
node.getNickname = function () {
if (node.nickname) {
return node.nickname
}
return
// return getNickname(node, node.comfyClass.trim())
}
}
const orig = node.__proto__.onDrawForeground
node.onDrawForeground = function (ctx) {
drawBadge(node, orig, arguments)
}
node.badge_enabled = true
}
}
})
},
async loadedGraphNode (node, app) {
// console.log(
@@ -1643,41 +1189,3 @@ app.registerExtension({
} catch (error) {}
}
})
//获取当前显存
function fetchSystemStats () {
return new Promise(async (resolve, reject) => {
try {
const response = await fetch('/system_stats')
const data = await response.json()
resolve(data)
} catch (error) {
reject(error)
}
})
}
//清理显存
function postFreeData () {
return new Promise(async (resolve, reject) => {
try {
const postData = {
unload_models: true,
free_memory: true
}
const response = await fetch('/free', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(postData)
})
if (response.ok) {
resolve()
} else {
reject(new Error('Request failed'))
}
} catch (error) {
reject(error)
}
})
}
+4 -1
View File
@@ -1,5 +1,8 @@
import { app } from '../../../scripts/app.js'
import { $el } from '../../../scripts/ui.js'
import { api } from '../../../scripts/api.js'
import { ComfyWidgets } from '../../../scripts/widgets.js'
import { $el } from '../../../scripts/ui.js'
import { addValueControlWidget } from '../../../scripts/widgets.js'
const getLocalData = key => {
let data = {}
-515
View File
@@ -1,515 +0,0 @@
import { app } from '../../../scripts/app.js'
import { api } from '../../../scripts/api.js'
import { ComfyWidgets } from '../../../scripts/widgets.js'
import { $el } from '../../../scripts/ui.js'
// The code is based on ComfyUI-VideoHelperSuite modification.
function injectCSS (css) {
// 检查页面中是否已经存在具有相同内容的style标签
const existingStyle = document.querySelector('style')
if (existingStyle && existingStyle.textContent === css) {
return // 如果已经存在相同的样式,则不进行注入
}
// 创建一个新的style标签,并将CSS内容注入其中
const style = document.createElement('style')
style.textContent = css
// 将style标签插入到页面的head元素中
const head = document.querySelector('head')
head.appendChild(style)
}
injectCSS(`
.hidden{
display:none !important
}`)
function get_position_style (ctx, widget_width, y, node_height) {
const MARGIN = 4 // the margin around the html element
/* Create a transform that deals with all the scrolling and zooming */
const elRect = ctx.canvas.getBoundingClientRect()
const transform = new DOMMatrix()
.scaleSelf(
elRect.width / ctx.canvas.width,
elRect.height / ctx.canvas.height
)
.multiplySelf(ctx.getTransform())
.translateSelf(MARGIN, MARGIN + y)
return {
transformOrigin: '0 0',
transform: transform,
left: `0`,
top: `0`,
cursor: 'pointer',
position: 'absolute',
maxWidth: `${widget_width - MARGIN * 2}px`,
// maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
width: `${widget_width - MARGIN * 2}px`,
// height: `${node_height * 0.3 - MARGIN * 2}px`,
// background: '#EEEEEE',
display: 'flex',
flexDirection: 'column',
// alignItems: 'center',
justifyContent: 'space-around'
}
}
function videoUpload (node, inputName, inputData, app) {
const imageWidget = node.widgets.find(w => w.name === 'video')
let uploadWidget
const widget = {
type: 'div',
name: 'upload-preview',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, 220, node.size[1]),
{
outline: '1px solid'
}
)
}
}
widget.div = $el('div', {})
widget.div.style.width = `120px`
document.body.appendChild(widget.div)
node.addCustomWidget(widget)
// console.log('#imageWidget', imageWidget)
const displayDiv = document.createElement('video')
displayDiv.controls = true
// displayDiv.style=`width:200px;height:200px`
imageWidget.callback = () => {
displayDiv.src = `/view?filename=${
imageWidget.value
}&type=input&subfolder=${''}&rand=${Math.random()}`
// displayDiv.onloadedmetadata = function () {
// var frameCount = displayDiv.duration * displayDiv.webkitDecodedFrameCount
// console.log('视频帧数:' + frameCount)
// node.widgets.filter(w => w.name == 'video_segment_frames')[0].value =
// frameCount
// }
}
if (imageWidget.value) {
// console.log(imageWidget.value)
displayDiv.src = `/view?filename=${
imageWidget.value
}&type=input&subfolder=${''}&rand=${Math.random()}`
}
widget.div.appendChild(displayDiv)
const onRemoved = node.onRemoved
node.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
var default_value = imageWidget.value
Object.defineProperty(imageWidget, 'value', {
set: function (value) {
this._real_value = value
},
get: function () {
let value = ''
if (this._real_value) {
value = this._real_value
} else {
return default_value
}
if (value.filename) {
let real_value = value
value = ''
if (real_value.subfolder) {
value = real_value.subfolder + '/'
}
value += real_value.filename
if (real_value.type && real_value.type !== 'input')
value += ` [${real_value.type}]`
}
return value
}
})
async function uploadFile (file, updateNode, pasted = false) {
try {
// Wrap file in formdata so it includes filename
const body = new FormData()
body.append('image', file)
if (pasted) body.append('subfolder', 'pasted')
const resp = await api.fetchApi('/upload/image', {
method: 'POST',
body
})
if (resp.status === 200) {
const data = await resp.json()
// Add the file to the dropdown list and update the widget value
let path = data.name
if (data.subfolder) path = data.subfolder + '/' + path
if (!imageWidget.options.values.includes(path)) {
imageWidget.options.values.push(path)
}
if (updateNode) {
imageWidget.value = path
}
return `/view?filename=${path}&type=input&subfolder=${
pasted ? 'pasted' : ''
}&rand=${Math.random()}`
} else {
alert(resp.status + ' - ' + resp.statusText)
}
} catch (error) {
alert(error)
}
}
const fileInput = document.createElement('input')
Object.assign(fileInput, {
type: 'file',
accept: 'video/*,.mkv,video/webm,video/mp4,video/x-matroska,image/gif',
style: 'display: none',
onchange: async () => {
if (fileInput.files.length) {
let file = fileInput.files[0]
const url = await uploadFile(file, true)
// console.log('fileInput', file)
var reader = new FileReader()
reader.onload = function () {
displayDiv.src = url
displayDiv.onloadedmetadata = function () {
// var frameCount =
// displayDiv.duration * displayDiv.webkitDecodedFrameCount
// console.log('视频帧数:' + frameCount)
// node.widgets.filter(
// w => w.name == 'video_segment_frames'
// )[0].value = frameCount
}
}
reader.readAsDataURL(file)
}
}
})
document.body.append(fileInput)
// Create the button widget for selecting the files
uploadWidget = node.addWidget('button', 'upload file', 'video', () => {
fileInput.click()
})
uploadWidget.serialize = false
return { widget: uploadWidget }
}
ComfyWidgets.VIDEOUPLOAD_ = videoUpload
app.registerExtension({
name: 'Mixlab.Video.LoadVideoAndSegment_',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeData?.name == 'LoadVideoAndSegment_') {
nodeData.input.required.upload = ['VIDEOUPLOAD_']
}
},
async loadedGraphNode (node, app) {
if (node.type === 'LoadVideoAndSegment_') {
const imageWidget = node.widgets.find(w => w.name === 'video')
const uploadPreview = node.widgets.find(w => w.name === 'upload-preview')
if (imageWidget.value) {
// console.log(imageWidget.value)
uploadPreview.div.querySelector('video').src = `/view?filename=${
imageWidget.value
}&type=input&subfolder=${''}&rand=${Math.random()}`
}
}
}
})
function offsetDOMWidget(
widget,
ctx,
node,
widgetWidth,
widgetY,
height
) {
const margin = 10
const elRect = ctx.canvas.getBoundingClientRect()
const transform = new DOMMatrix()
.scaleSelf(
elRect.width / ctx.canvas.width,
elRect.height / ctx.canvas.height
)
.multiplySelf(ctx.getTransform())
.translateSelf(0, widgetY + margin)
const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
Object.assign(widget.inputEl.style, {
transformOrigin: '0 0',
transform: scale,
left: `${transform.e}px`,
top: `${transform.d + transform.f}px`,
width: `${widgetWidth}px`,
height: `${(height || widget.parent?.inputHeight || 32) - margin}px`,
position: 'absolute',
background: !node.color ? '' : node.color,
color: !node.color ? '' : 'white',
zIndex: 5, //app.graph._nodes.indexOf(node),
})
}
export const hasWidgets = (node) => {
if (!node.widgets || !node.widgets?.[Symbol.iterator]) {
return false
}
return true
}
export const cleanupNode = (node) => {
if (!hasWidgets(node)) {
return
}
for (const w of node.widgets) {
if (w.canvas) {
w.canvas.remove()
}
if (w.inputEl) {
w.inputEl.remove()
}
// calls the widget remove callback
w.onRemoved?.()
}
}
const CreatePreviewElement = (name, val, format) => {
const [type] = format.split('/')
const w = {
name,
type,
value: val,
draw: function (ctx, node, widgetWidth, widgetY, height) {
const [cw, ch] = this.computeSize(widgetWidth)
offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, ch)
},
computeSize: function (_) {
const ratio = this.inputRatio || 1
const width = Math.max(220, this.parent.size[0])
return [width, (width / ratio + 10)]
},
onRemoved: function () {
if (this.inputEl) {
this.inputEl.remove()
}
},
}
w.inputEl = document.createElement(type === 'video' ? 'video' : 'img')
w.inputEl.src = w.value
if (type === 'video') {
w.inputEl.setAttribute('type', 'video/webm');
w.inputEl.autoplay = true
w.inputEl.loop = true
w.inputEl.controls = false;
}
w.inputEl.onload = function () {
w.inputRatio = w.inputEl.naturalWidth / w.inputEl.naturalHeight
}
document.body.appendChild(w.inputEl)
return w
}
app.registerExtension({
name: 'Mixlab.Video.ImageListReplace',
async beforeRegisterNodeDef (nodeType, nodeData, app) {
if (nodeData?.name == 'ImageListReplace_') {
const orig_nodeCreated = nodeType.prototype.onNodeCreated
nodeType.prototype.onNodeCreated = function () {
orig_nodeCreated?.apply(this, arguments)
const widget = {
type: 'div',
name: 'preview',
draw (ctx, node, widget_width, y, widget_height) {
Object.assign(
this.div.style,
get_position_style(ctx, widget_width, 188, node.size[1]),
{
outline: '1px solid',
display: 'flex',
flexWrap: 'wrap',
flexDirection: 'row',
justifyContent: 'flex-start'
}
)
}
}
widget.div = $el('div', {})
widget.div.style.width = `120px`
widget.div.className = 'hidden'
document.body.appendChild(widget.div)
this.addCustomWidget(widget)
// console.log('#ImageListReplace', widget)
const onRemoved = this.onRemoved
this.onRemoved = () => {
widget.div.remove()
return onRemoved?.()
}
}
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments)
// let _image_replace = message._image_replace[0]
// _image_replace = `/view?filename=${_image_replace.filename}&type=${
// _image_replace.type
// }&subfolder=${_image_replace.subfolder}&rand=${Math.random()}`
let preview = this.widgets.filter(w => w.name == 'preview')[0]
if (message._images.length > 0) {
preview.div.className = ''
// console.log('#ImageListReplace', preview.div)
}
preview.div.innerHTML = ''
for (const img_ of message._images) {
let img = new Image()
img.style = `width: 100px;
margin: 4px;`
img.src = `/view?filename=${img_.filename}&type=${
img_.type
}&subfolder=${img_.subfolder}&rand=${Math.random()}`
preview.div.appendChild(img)
}
let start_index = this.widgets.filter(w => w.name == 'start_index')[0]
let end_index = this.widgets.filter(w => w.name == 'end_index')[0]
let invert = this.widgets.filter(w => w.name == 'invert')[0]
let _sc = start_index.callback.bind(start_index)
let _ec = end_index.callback.bind(end_index)
const selectImages = () => {
// console.log(v)
let s = start_index.value,
e = end_index.value
let imgs = preview.div.querySelectorAll('img')
for (let index = 0; index < imgs.length; index++) {
if (invert.value) {
imgs[index].style.outline =
index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
} else {
imgs[index].style.outline =
index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
}
}
}
selectImages()
start_index.callback = v => {
let s = v,
e = end_index.value
let imgs = preview.div.querySelectorAll('img')
for (let index = 0; index < imgs.length; index++) {
if (invert.value) {
imgs[index].style.outline =
index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
} else {
imgs[index].style.outline =
index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
}
}
_sc(v)
}
end_index.callback = v => {
let s = start_index.value,
e = v
let imgs = preview.div.querySelectorAll('img')
for (let index = 0; index < imgs.length; index++) {
if (invert.value) {
imgs[index].style.outline =
index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
} else {
imgs[index].style.outline =
index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
}
}
_ec(v)
}
invert.callback = v => {
selectImages()
}
try {
} catch (error) {}
}
}
if (nodeData?.name == 'VideoCombine_Adv') {
const onExecuted = nodeType.prototype.onExecuted
nodeType.prototype.onExecuted = function (message) {
const prefix = 'vhs_gif_preview_'
const r = onExecuted ? onExecuted.apply(this, message) : undefined
if (this.widgets) {
const pos = this.widgets.findIndex(w => w.name === `${prefix}_0`)
if (pos !== -1) {
for (let i = pos; i < this.widgets.length; i++) {
this.widgets[i].onRemoved?.()
}
this.widgets.length = pos
}
if (message?.gifs) {
message.gifs.forEach((params, i) => {
const previewUrl = api.apiURL(
'/view?' + new URLSearchParams(params).toString()
)
const w = this.addCustomWidget(
CreatePreviewElement(
`${prefix}_${i}`,
previewUrl,
params.format || 'image/gif'
)
)
w.parent = this
})
}
const onRemoved = this.onRemoved
this.onRemoved = () => {
cleanupNode(this)
return onRemoved?.()
}
}
this.setSize([
this.size[0],
this.computeSize([this.size[0], this.size[1]])[1]
])
return r
}
}
}
})
+64 -81
View File
File diff suppressed because one or more lines are too long
+24
View File
@@ -0,0 +1,24 @@
::-webkit-scrollbar {
width: 2px;
}
@keyframes loading_mixlab {
0% {
background-color: green;
}
50% {
background-color: lightgreen;
}
100% {
background-color: green;
}
}
.loading_mixlab {
background-color: green;
animation-name: loading_mixlab;
animation-duration: 2s;
animation-iteration-count: infinite;
}
+9 -10
View File
@@ -1,6 +1,6 @@
{
"last_node_id": 24,
"last_link_id": 26,
"last_node_id": 23,
"last_link_id": 25,
"nodes": [
{
"id": 9,
@@ -203,7 +203,6 @@
"Node name for S&R": "ShowTextForGPT"
},
"widgets_values": [
"a girl face,super,(Pop Art:1.26),(Black and White:1.26)",
"a girl face,super,(Pop Art:1.26),(Black and White:1.26)"
]
},
@@ -538,7 +537,7 @@
"type": "IMAGE",
"links": [
24,
26
25
],
"slot_index": 0
}
@@ -548,7 +547,7 @@
}
},
{
"id": 24,
"id": 23,
"type": "AppInfo",
"pos": [
3363.0014990624995,
@@ -563,9 +562,9 @@
"mode": 0,
"inputs": [
{
"name": "IMAGE",
"name": "LOGO",
"type": "IMAGE",
"link": 26
"link": 25
}
],
"properties": {
@@ -581,7 +580,7 @@
"https://",
"",
"enable",
2
1
]
}
],
@@ -707,10 +706,10 @@
"IMAGE"
],
[
26,
25,
10,
0,
24,
23,
0,
"IMAGE"
]
File diff suppressed because one or more lines are too long