commit BooleanOperator, NumberCalculator, TextBox, Integer, Float, Boolean, QWenImage2Prompt

This commit is contained in:
chflame
2024-04-30 16:52:43 +08:00
parent 58afd31d1b
commit 08018e2f6e
14 changed files with 723 additions and 2 deletions
+48
View File
@@ -70,6 +70,9 @@ When this error has occurred, please check the network environment.
## Update
<font size="4">**If the dependency package error after updating, please reinstall the relevant dependency packages. </font><br />
* Commit [QWenImage2Prompt](#QWenImage2Prompt) node, this node is repackage of the [ComfyUI_VLM_nodes](https://github.com/gokayfem/ComfyUI_VLM_nodes)'s ```UForm-Gen2 Qwen Node```, thanks to the original author.
* Commit [BooleanOperator](#BooleanOperator), [NumberCalculator](#NumberCalculator), [TextBox](#TextBox), [Integer](#Integer), [Float](#Float), [Boolean](#Boolean) nodes. These nodes can perform mathematical and logical operations.
* Commit [ExtendCanvasV2](#ExtendCanvasV2) node,support color value input.
* Commit [AutoBrightness](#AutoBrightness) node,it can automatically adjust the brightness of image.
* [CreateGradientMask](#CreateGradientMask) node add ```center``` option.
* Commit [GetColorToneV2](#GetColorToneV2) node, can select the main and average colors for the background or body.
@@ -562,6 +565,19 @@ The following changes have been made based on ImageScaleByAspectRatio:
* scale_to_side: Allow scaling by specified size on long or short sides.
* scale_to_length: When scale_by_side is set to "longest", this will be used as the length of the long edge of the image; When set to "shortest", it serves as the length of the short edge.
### <a id="table1">QWenImage2Prompt</a>
Inference the prompts based on the image. this node is repackage of the [ComfyUI_VLM_nodes](https://github.com/gokayfem/ComfyUI_VLM_nodes)'s ```UForm-Gen2 Qwen Node```, thanks to the original author.
Download model files from [huggingface](https://huggingface.co/unum-cloud/uform-gen2-qwen-500m) or [Baidu Netdisk](https://pan.baidu.com/s/1ztnVX_Sh800xsWZhMDe-Ww?pwd=esyt) to ```ComfyUI/models/LLavacheckpoints/files_for_uform_gen2_qwen``` folder.
![image](image/qwen_image2prompt_example.png)
Node Options:
* question: Prompt of UForm-Gen-QWen model.
### <a id="table1">PromptTagger</a>
Inference the prompts based on the image. it can replace key word for the prompt. This node currently uses Google Gemini API as the backend service. Please ensure that the network environment can use Gemini normally.
Please apply for your API key on [Google AI Studio](https://makersuite.google.com/app/apikey), And fill it in ```api_key.ini```, this file is located in the root directory of the plug-in, and the default name is ```api_key.ini.example```. to use this file for the first time, you need to change the file suffix to ```.ini```. Open it using text editing software, fill in your API key after ```google_api_key=``` and save it.
@@ -958,6 +974,38 @@ Used to provide assistance for workflow debugging. When running, the properties
This node allows any type of input.
### <a id="table1">TextBox</a>
![image](image/text_box_node.png)
Output a string.
### <a id="table1">Integer</a>
![image](image/integer_node.png)
Output a integer value.
### <a id="table1">Float</a>
![image](image/float_node.png)
Output a floating-point value with a precision of 5 decimal places.
### <a id="table1">Boolean</a>
![image](image/boolean_node.png)
Output a boolean value.
### <a id="table1">NumberCalculator</a>
![image](image/number_calculator_node.png)
Performs mathematical operations on two numeric values and outputs integer and floating point results<sup>*</sup>. Supported operations include```+```, ```-```, ```*```, ```/```, ```**```, ```//```, ```%```.
<sup>*</sup> The input only supports boolean, integer, and floating point numbers, forcing in other data will result in error.
### <a id="table1">BooleanOperator</a>
![image](image/boolean_operator_node.png)
Perform a Boolean operation on two numeric values and output the result<sup>*</sup>. Supported operations include```==```, ```!=```, ```and```, ```or```, ```xor```, ```not```.
<sup>*</sup> The input only supports boolean, integer, and floating point numbers, forcing in other data will result in error. The ```and``` operation between the values outputs a larger number, and the ```or``` operation outputs a smaller number.
# <a id="table1">LayerMask</a>
![image](image/layermask_nodes.png)
+42
View File
@@ -70,6 +70,9 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git
## 更新说明
<font size="4">**如果本插件更新后出现依赖包错误,请重新安装相关依赖包。
* 添加 [QWenImage2Prompt](#QWenImage2Prompt)节点, 用本地模型反推提示词。(需要下载模型到models文件夹)。这个节点是[ComfyUI_VLM_nodes](https://github.com/gokayfem/ComfyUI_VLM_nodes)中的```UForm-Gen2 Qwen Node```节点的重新封装,感谢原作者。
* 添加 [BooleanOperator](#BooleanOperator), [NumberCalculator](#NumberCalculator), [TextBox](#TextBox), [Integer](#Integer), [Float](#Float), [Boolean](#Boolean) 节点。这些节点可进行数学和逻辑运算。
* 添加 [ExtendCanvasV2](#ExtendCanvasV2) 节点,支持color值输入。
* 添加 [AutoBrightness](#AutoBrightness) 节点,可自动调整图片亮度。
* [CreateGradientMask](#CreateGradientMask) 节点增加 ```center``` 选项。
* 添加 [GetColorToneV2](#GetColorToneV2) 节点, 可选择背景或主体的主色和平均色。
@@ -559,6 +562,17 @@ ImageScaleByAspectRatio的V2升级版
* scale_to_length: scale_by_side被设置为"longest"时,此项将作为是图像长边的长度; 设置为"shortest"时,作为短边的长度。
### <a id="table1">QWenImage2Prompt</a>
根据图片反推提示词。这个节点是[ComfyUI_VLM_nodes](https://github.com/gokayfem/ComfyUI_VLM_nodes)中的```UForm-Gen2 Qwen Node```节点的重新封装,感谢原作者。
请从[huggingface](https://huggingface.co/unum-cloud/uform-gen2-qwen-500m)或者[百度网盘](https://pan.baidu.com/s/1ztnVX_Sh800xsWZhMDe-Ww?pwd=esyt)下载模型到```ComfyUI/models/LLavacheckpoints/files_for_uform_gen2_qwen```文件夹。
![image](image/qwen_image2prompt_example.png)
节点选项说明:
* question: 对UForm-Gen-QWen模型的提示词。
### <a id="table1">PromptTagger</a>
根据图片反推提示词,可以设置替换词。这个节点目前使用Google Gemini API作为后端服务,请确保网络环境可以正常使用Gemini。
请在[Google AI Studio](https://makersuite.google.com/app/apikey)申请你的API key, 并将其填到```api_key.ini```, 这个文件位于插件根目录下, 默认名字是```api_key.ini.example```, 初次使用这个文件需将文件后缀改为.ini。用文本编辑软件打开,在```google_api_key=```后面填入你的API key并保存。
@@ -955,6 +969,34 @@ cropped_mask: 裁切后的遮罩。
这个节点允许任意类型的输入。
### <a id="table1">TextBox</a>
![image](image/text_box_node.png)
输出一段字符串。
### <a id="table1">Integer</a>
![image](image/integer_node.png)
输出一个整数。
### <a id="table1">Float</a>
![image](image/float_node.png)
输出一个浮点数,精度是小数点后5位。
### <a id="table1">Boolean</a>
![image](image/boolean_node.png)
输出一个布尔值。
### <a id="table1">NumberCalculator</a>
![image](image/number_calculator_node.png)
对两个数值进行数学运算并输出整数和浮点数结果<sup>*</sup>。支持的运算包括```+```、```-```、```*```、```/```、```**```、```//```、```%```。
<sup>*</sup> 输入仅支持布尔值、整数和浮点数,强行接入其他数据将导致错误。
### <a id="table1">BooleanOperator</a>
![image](image/boolean_operator_node.png)
对两个数值进行布尔运算并输出结果<sup>*</sup>。支持的运算包括```==```、```!=```、```and```、```or```、```xor```、```not```。
<sup>*</sup> 输入仅支持布尔值、整数和浮点数,强行接入其他数据将导致错误。数值之间的```and```运算输出较大的数,```or```运算输出较小的数。
# <a id="table1">LayerMask</a>
Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 460 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 716 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

+114
View File
@@ -0,0 +1,114 @@
import os.path
from pathlib import Path
from transformers import AutoModel, AutoProcessor, StoppingCriteria, StoppingCriteriaList
import torch
from PIL import Image
from torchvision.transforms import ToPILImage
from huggingface_hub import snapshot_download
import folder_paths
# Define the directory for saving files related to uform-gen2-qwen
# files_for_uform_gen2_qwen = Path(folder_paths.folder_names_and_paths["LLavacheckpoints"][0][0]) / "files_for_uform_gen2_qwen"
files_for_uform_gen2_qwen = Path(os.path.join(folder_paths.models_dir, "LLavacheckpoints", "files_for_uform_gen2_qwen"))
files_for_uform_gen2_qwen.mkdir(parents=True, exist_ok=True) # Ensure the directory exists
class StopOnTokens(StoppingCriteria):
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
stop_ids = [151645] # Define stop tokens as per your model's specifics
for stop_id in stop_ids:
if input_ids[0][-1] == stop_id:
return True
return False
class UformGen2QwenChat:
def __init__(self):
self.model_path = snapshot_download("unum-cloud/uform-gen2-qwen-500m",
local_dir=files_for_uform_gen2_qwen,
force_download=False, # Set to True if you always want to download, regardless of local copy
local_files_only=False, # Set to False to allow downloading if not available locally
local_dir_use_symlinks="auto") # or set to True/False based on your symlink preference
self.device = "cuda:0" if torch.cuda.is_available() else "cpu"
self.model = AutoModel.from_pretrained(self.model_path, trust_remote_code=True).to(self.device)
self.processor = AutoProcessor.from_pretrained(self.model_path, trust_remote_code=True)
def chat_response(self, message, history, image_path):
stop = StopOnTokens()
messages = [{"role": "system", "content": "You are a helpful Assistant."}]
for user_msg, assistant_msg in history:
messages.append({"role": "user", "content": user_msg})
messages.append({"role": "assistant", "content": assistant_msg})
if len(messages) == 1:
message = f" <image>{message}"
messages.append({"role": "user", "content": message})
model_inputs = self.processor.tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
)
image = Image.open(image_path) # Load image using PIL
image_tensor = (
self.processor.feature_extractor(image)
.unsqueeze(0)
)
attention_mask = torch.ones(
1, model_inputs.shape[1] + self.processor.num_image_latents - 1
)
model_inputs = {
"input_ids": model_inputs,
"images": image_tensor,
"attention_mask": attention_mask
}
model_inputs = {k: v.to(self.device) for k, v in model_inputs.items()}
output = self.model.generate(
**model_inputs,
max_new_tokens=1024,
stopping_criteria=StoppingCriteriaList([stop])
)
response_text = self.processor.tokenizer.decode(output[0], skip_special_tokens=True)
return response_text
# Example of integrating UformGen2QwenChat into a node-like structure
class QWenImage2Prompt:
def __init__(self):
self.chat_model = UformGen2QwenChat()
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"image": ("IMAGE",),
"question": ("STRING", {"multiline": False, "default": "describe this image",},),
},
}
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("text",)
FUNCTION = "uform_gen2_qwen_chat"
CATEGORY = '😺dzNodes/LayerUtility'
def uform_gen2_qwen_chat(self, image, question):
history = [] # Example empty history
pil_image = ToPILImage()(image[0].permute(2, 0, 1))
temp_path = files_for_uform_gen2_qwen / "temp.png"
pil_image.save(temp_path)
response = self.chat_model.chat_response(question, history, temp_path)
return (response.split("assistant\n", 1)[1], )
NODE_CLASS_MAPPINGS = {
"LayerUtility: QWenImage2Prompt": QWenImage2Prompt
}
NODE_DISPLAY_NAME_MAPPINGS = {
"LayerUtility: QWenImage2Prompt": "LayerUtility: QWenImage2Prompt"
}
+163
View File
@@ -0,0 +1,163 @@
from .imagefunc import AnyType
any = AnyType("*")
class BooleanOperator:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
operator_list = ["==", "!=", "and", "or", "xor", "not(a)"]
return {"required": {
"a": (any, {}),
"b": (any, {}),
"operator": (operator_list,),
},}
RETURN_TYPES = ("BOOLEAN",)
RETURN_NAMES = ("boolean",)
FUNCTION = 'bool_operator_node'
CATEGORY = '😺dzNodes/LayerUtility'
def bool_operator_node(self, a, b, operator):
ret_value = False
if operator == "==":
ret_value = a == b
if operator == "!=":
ret_value = a != b
if operator == "and":
ret_value = a and b
if operator == "or":
ret_value = a or b
if operator == "xor":
ret_value = not(a == b)
if operator == "not(a)":
ret_value = not a
return (ret_value,)
class NumberCalculator:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
operator_list = ["+", "-", "*", "/", "**", "//", "%" ]
return {"required": {
"a": (any, {}),
"b": (any, {}),
"operator": (operator_list,),
},}
RETURN_TYPES = ("INT", "FLOAT",)
RETURN_NAMES = ("int", "float",)
FUNCTION = 'number_calculator_node'
CATEGORY = '😺dzNodes/LayerUtility'
def number_calculator_node(self, a, b, operator):
ret_value = 0
if operator == "+":
ret_value = a + b
if operator == "-":
ret_value = a - b
if operator == "*":
ret_value = a * b
if operator == "/":
ret_value = a / b
if operator == "**":
ret_value = a ** b
if operator == "//":
ret_value = a // b
if operator == "%":
ret_value = a % b
return (int(ret_value), float(ret_value),)
class TextBoxNode:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
return {"required": {
"text": ("STRING", {"multiline": True}),
},}
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("text",)
FUNCTION = 'text_box_node'
CATEGORY = '😺dzNodes/LayerUtility'
def text_box_node(self, text):
return (text,)
class IntegerNode:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
return {"required": {
"int_value":("INT", {"default": 0, "min": -99999999999999999999, "max": 99999999999999999999, "step": 1}),
},}
RETURN_TYPES = ("INT",)
RETURN_NAMES = ("int",)
FUNCTION = 'integer_node'
CATEGORY = '😺dzNodes/LayerUtility'
def integer_node(self, int_value):
return (int_value,)
class FloatNode:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
return {"required": {
"float_value": ("FLOAT", {"default": 0, "min": -99999999999999999999, "max": 99999999999999999999, "step": 0.00001}),
},}
RETURN_TYPES = ("FLOAT",)
RETURN_NAMES = ("float",)
FUNCTION = 'float_node'
CATEGORY = '😺dzNodes/LayerUtility'
def float_node(self, float_value):
return (float_value,)
class BooleanNode:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(self):
return {"required": {
"bool_value": ("BOOLEAN", {"default": False}),
},}
RETURN_TYPES = ("BOOLEAN",)
RETURN_NAMES = ("boolean",)
FUNCTION = 'boolean_node'
CATEGORY = '😺dzNodes/LayerUtility'
def boolean_node(self, bool_value):
return (bool_value,)
NODE_CLASS_MAPPINGS = {
"LayerUtility: BooleanOperator": BooleanOperator,
"LayerUtility: NumberCalculator": NumberCalculator,
"LayerUtility: TextBox": TextBoxNode,
"LayerUtility: Integer": IntegerNode,
"LayerUtility: Float": FloatNode,
"LayerUtility: Boolean": BooleanNode
}
NODE_DISPLAY_NAME_MAPPINGS = {
"LayerUtility: BooleanOperator": "LayerUtility: BooleanOperator",
"LayerUtility: NumberCalculator": "LayerUtility: NumberCalculator",
"LayerUtility: TextBox": "LayerUtility: TextBox",
"LayerUtility: Integer": "LayerUtility: Integer",
"LayerUtility: Float": "LayerUtility: Float",
"LayerUtility: Boolean": "LayerUtility: Boolean"
}
+7 -2
View File
@@ -16,11 +16,12 @@ class PrintInfo:
CATEGORY = '😺dzNodes/LayerUtility'
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("info",)
RETURN_NAMES = ("text",)
FUNCTION = "print_info"
OUTPUT_NODE = True
def print_info(self, anything=None):
value = f'PrintInfo: \n Input type = {type(anything)}'
value = f'PrintInfo:\nInput type = {type(anything)}'
if isinstance(anything, torch.Tensor):
value += f"\n Input dim = {anything.dim()}, shape[0] = {anything.shape[0]} \n"
for i in range(anything.shape[0]):
@@ -38,7 +39,11 @@ class PrintInfo:
value = value + str(anything) + "\n"
except Exception:
value = 'source exists, but could not be serialized.'
else:
value = 'source does not exist.'
log(value)
return (value,)
NODE_CLASS_MAPPINGS = {
+349
View File
@@ -0,0 +1,349 @@
{
"last_node_id": 11,
"last_link_id": 12,
"nodes": [
{
"id": 1,
"type": "LayerUtility: Integer",
"pos": [
843,
477
],
"size": {
"0": 247.19744873046875,
"1": 61.07960510253906
},
"flags": {},
"order": 0,
"mode": 0,
"outputs": [
{
"name": "int",
"type": "INT",
"links": [
1
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerUtility: Integer"
},
"widgets_values": [
9
]
},
{
"id": 7,
"type": "Display Any (rgthree)",
"pos": [
1498,
452
],
"size": {
"0": 210,
"1": 76.0000228881836
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 5,
"dir": 3
}
],
"properties": {
"Node name for S&R": "Display Any (rgthree)"
},
"widgets_values": [
""
]
},
{
"id": 8,
"type": "Display Any (rgthree)",
"pos": [
1499,
564
],
"size": {
"0": 210.36199951171875,
"1": 76
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 6,
"dir": 3
}
],
"properties": {
"Node name for S&R": "Display Any (rgthree)"
},
"widgets_values": [
""
]
},
{
"id": 5,
"type": "LayerUtility: NumberCalculator",
"pos": [
1168,
534
],
"size": {
"0": 252,
"1": 83.83000183105469
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "a",
"type": "*",
"link": 1
},
{
"name": "b",
"type": "*",
"link": 2
}
],
"outputs": [
{
"name": "int",
"type": "INT",
"links": [
5
],
"shape": 3,
"slot_index": 0
},
{
"name": "float",
"type": "FLOAT",
"links": [
6
],
"shape": 3,
"slot_index": 1
}
],
"properties": {
"Node name for S&R": "LayerUtility: NumberCalculator"
},
"widgets_values": [
"*"
]
},
{
"id": 3,
"type": "LayerUtility: Float",
"pos": [
841,
627
],
"size": {
"0": 245.2032928466797,
"1": 60.5140495300293
},
"flags": {},
"order": 1,
"mode": 0,
"outputs": [
{
"name": "float",
"type": "FLOAT",
"links": [
2,
11
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerUtility: Float"
},
"widgets_values": [
2.0331300000000003
]
},
{
"id": 11,
"type": "LayerUtility: Float",
"pos": [
841,
794
],
"size": {
"0": 245.2032928466797,
"1": 60.5140495300293
},
"flags": {},
"order": 2,
"mode": 0,
"outputs": [
{
"name": "float",
"type": "FLOAT",
"links": [
12
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerUtility: Float"
},
"widgets_values": [
2.0331300000000003
]
},
{
"id": 10,
"type": "LayerUtility: BooleanOperator",
"pos": [
1175,
739
],
"size": {
"0": 250.57093811035156,
"1": 92.19929504394531
},
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "a",
"type": "*",
"link": 11
},
{
"name": "b",
"type": "*",
"link": 12
}
],
"outputs": [
{
"name": "boolean",
"type": "BOOLEAN",
"links": [
10
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerUtility: BooleanOperator"
},
"widgets_values": [
"=="
]
},
{
"id": 9,
"type": "Display Any (rgthree)",
"pos": [
1500,
754
],
"size": {
"0": 210,
"1": 76.0000228881836
},
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 10,
"dir": 3
}
],
"properties": {
"Node name for S&R": "Display Any (rgthree)"
},
"widgets_values": [
""
]
}
],
"links": [
[
1,
1,
0,
5,
0,
"*"
],
[
2,
3,
0,
5,
1,
"*"
],
[
5,
5,
0,
7,
0,
"*"
],
[
6,
5,
1,
8,
0,
"*"
],
[
10,
10,
0,
9,
0,
"*"
],
[
11,
3,
0,
10,
0,
"*"
],
[
12,
11,
0,
10,
1,
"*"
]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}