diff --git a/README.md b/README.md index 6806efb..31af6e5 100644 --- a/README.md +++ b/README.md @@ -144,6 +144,7 @@ Please try downgrading the ```protobuf``` dependency package to 3.20.3, or set e **If the dependency package error after updating, please double clicking ```repair_dependency.bat``` (for Official ComfyUI Protable) or ```repair_dependency_aki.bat``` (for ComfyUI-aki-v1.x) in the plugin folder to reinstall the dependency packages. +* Commit [JimengImageToImageAPI](#JimengImageToImageAPI) node, edit images using the Instant Dreaming Image 3.0 API. Create an account on [Volcano Engine](#https://console.volcengine.com/iam/keymanage) and apply for API AccessKeyID and SecretAccessKey. Fill them into the ```api_key.ini``` directory in the plugin directory. * Commit [SAM2UltraV2](SAM2UltraV2) and [LoadSAM2Model](LoadSAM2Model) nodes, Change the SAM model to an external input to save resources when using multiple nodes. * Commit [JoyCaptionBetaOne](JoyCaptionBetaOne), [LoadJoyCaptionBeta1Model](LoadJoyCaptionBeta1Model), [JoyCaptionBeta1ExtraOptions](JoyCaptionBeta1ExtraOptions) nodes, Generate prompt words using the JoyCaption Beta One model. * Commit [SaveImagePLusV2](SaveImagePlusV2) node, add custom file names and setting up the dpi of image. @@ -563,6 +564,20 @@ Node Options: * dtype: The model accuracy has two options: bf16 and fp32. * device: The model loading device has two options: cuda or cpu. +### JimengImageToImageAPI +Edit images using the Jimeng API. +Please create an account on [Volcano Engine](#https://console.volcengine.com/iam/keymanage), apply for API AccessKeyID and SecretAccessKey, and fill them in ```api_key. ini```. This file is located in the root directory of the plugin and its default name is ```api_key.ini.example```. When using this file for the first time, you need to change the file extension to '.ini'. Open with text editing software and fill in the corresponding values after ```volcengine_SecretAccessKey=``` and ```volcengine_SecretAccessKey=```. +![image](image/jimeng_image_to_image_api_example.jpg) + +Node Options: +![image](image/jimeng_image_to_image_api_node.jpg) + +* image: The input image. +* model: Choose the DreamMap and Life Map model. Currently, only the Jimeng_i2i-v30 model is supported. +* time_out: The maximum time limit for waiting for API return, in seconds. If this time is exceeded, the node will end running. +* scale: The scale parameter of jimeng_i2iuv30 is set to 0.5 by default. +* seed: The seed value. +* prompt: The prompt. ### UserPromptGeneratorTxtImg diff --git a/README_CN.MD b/README_CN.MD index d618b8e..5c85d8e 100644 --- a/README_CN.MD +++ b/README_CN.MD @@ -121,6 +121,7 @@ If this call came from a _pb2.py file, your generated code is out of date and mu ## 更新说明 **如果本插件更新后出现依赖包错误，请双击运行插件目录下的```install_requirements.bat```(官方便携包)，或 ```install_requirements_aki.bat```(秋叶整合包) 重新安装依赖包。 +* 添加 [JimengImageToImageAPI](#JimengImageToImageAPI) 节点，使用即梦图生图3.0API对图片进行编辑。在[火山引擎](#https://console.volcengine.com/iam/keymanage) 创建账号，并申请API AccessKeyID 和 SecretAccessKey，将其填入插件目录下的```api_key.ini```。 * 添加 [SAM2UltraV2](SAM2UltraV2) 和 [LoadSAM2Model](LoadSAM2Model) 节点，将SAM模型改为外部输入，在使用多个节点时节省资源。 * 添加 [JoyCaptionBetaOne](JoyCaptionBetaOne), [LoadJoyCaptionBeta1Model](LoadJoyCaptionBeta1Model), [JoyCaptionBeta1ExtraOptions](JoyCaptionBeta1ExtraOptions) 节点，使用JoyCaption-Beta-One模型生成提示词。 * 添加 [SaveImagePLusV2](SaveImagePlusV2) 节点，增加自定义文件名和设置dpi。 @@ -539,6 +540,20 @@ JoyCaption Beta One的extra_options参数节点。 * dtype: 模型精度，有bf16和fp32两个选项。 * device: 模型加载设备，有cuda和cpu两个选项。 +### JimengImageToImageAPI +使用即梦图生图3.0API对图片进行编辑。 +请在[火山引擎](#https://console.volcengine.com/iam/keymanage) 创建账号，并申请API AccessKeyID 和 SecretAccessKey，并将其填到```api_key.ini```。 这个文件位于插件根目录下, 默认名字是```api_key.ini.example```, 初次使用这个文件需将文件后缀改为.ini。用文本编辑软件打开，在```volcengine_AccessKeyId=``` 和 ```volcengine_SecretAccessKey=```后面填入对应的值。 +![image](image/jimeng_image_to_image_api_example.jpg) + +节点选项说明: +![image](image/jimeng_image_to_image_api_node.jpg) + +* image: 图片输入。 +* model: 选择即梦图生图模型。目前仅支持 jimeng_i2i_v30 模型。 +* time_out: 等待API返回的时间上限，单位为秒，超出此时间将结束节点运行。 +* scale: jimeng_i2i_v30的scale参数，默认为0.5。 +* seed: 种子值。 +* prompt: 提示词。 ### UserPromptGeneratorTxtImg 用于生成SD文本到图片提示词的UserPrompt预设。 diff --git a/api_key.ini.example b/api_key.ini.example index 720382a..a810d03 100644 --- a/api_key.ini.example +++ b/api_key.ini.example @@ -2,5 +2,7 @@ google_api_key= zhipu_api_key= deepseek_api_key= -volcengine_api_key= aliyun_api_key= +volcengine_api_key= +volcengine_AccessKeyId= +volcengine_SecretAccessKey= \ No newline at end of file diff --git a/image/jimeng_image_to_image_api_example.jpg b/image/jimeng_image_to_image_api_example.jpg new file mode 100644 index 0000000..d8ebc91 Binary files /dev/null and b/image/jimeng_image_to_image_api_example.jpg differ diff --git a/image/jimeng_image_to_image_api_node.jpg b/image/jimeng_image_to_image_api_node.jpg new file mode 100644 index 0000000..3eccca4 Binary files /dev/null and b/image/jimeng_image_to_image_api_node.jpg differ diff --git a/py/Jimeng_API.py b/py/Jimeng_API.py new file mode 100644 index 0000000..f489f9c --- /dev/null +++ b/py/Jimeng_API.py @@ -0,0 +1,249 @@ +import json +import sys +import os +import io +import base64 +import datetime +import hashlib +import hmac +import requests +import time +from PIL import Image +import torch +from .imagefunc import tensor2pil, pil2tensor, log, get_api_key, fit_resize_image + +jimeng_i2i_model_list = ["jimeng_i2i_v30"] +jimeng_t2i_model_list = ["jimeng_high_aes_general_v21_L"] + + +method = 'POST' +host = 'visual.volcengineapi.com' +region = 'cn-north-1' +endpoint = 'https://visual.volcengineapi.com' +service = 'cv' +image_max_side_length = 4096 + +def sign(key, msg): + return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest() + +def getSignatureKey(key, dateStamp, regionName, serviceName): + kDate = sign(key.encode('utf-8'), dateStamp) + kRegion = sign(kDate, regionName) + kService = sign(kRegion, serviceName) + kSigning = sign(kService, 'request') + return kSigning + +def formatQuery(parameters): + request_parameters_init = '' + for key in sorted(parameters): + request_parameters_init += key + '=' + parameters[key] + '&' + request_parameters = request_parameters_init[:-1] + return request_parameters + + +def signV4Request(access_key, secret_key, service, req_query, req_body): + if access_key is None or secret_key is None: + print('No access key is available.') + sys.exit() + + t = datetime.datetime.utcnow() + current_date = t.strftime('%Y%m%dT%H%M%SZ') + # current_date = '20210818T095729Z' + datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope + canonical_uri = '/' + canonical_querystring = req_query + signed_headers = 'content-type;host;x-content-sha256;x-date' + payload_hash = hashlib.sha256(req_body.encode('utf-8')).hexdigest() + content_type = 'application/json' + canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + \ + '\n' + 'x-content-sha256:' + payload_hash + \ + '\n' + 'x-date:' + current_date + '\n' + canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + \ + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash + # print(canonical_request) + algorithm = 'HMAC-SHA256' + credential_scope = datestamp + '/' + region + '/' + service + '/' + 'request' + string_to_sign = algorithm + '\n' + current_date + '\n' + credential_scope + '\n' + hashlib.sha256( + canonical_request.encode('utf-8')).hexdigest() + # print(string_to_sign) + signing_key = getSignatureKey(secret_key, datestamp, region, service) + # print(signing_key) + signature = hmac.new(signing_key, (string_to_sign).encode( + 'utf-8'), hashlib.sha256).hexdigest() + # print(signature) + + authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + \ + credential_scope + ', ' + 'SignedHeaders=' + \ + signed_headers + ', ' + 'Signature=' + signature + # print(authorization_header) + headers = {'X-Date': current_date, + 'Authorization': authorization_header, + 'X-Content-Sha256': payload_hash, + 'Content-Type': content_type + } + # print(headers) + + # ************* SEND THE REQUEST ************* + request_url = endpoint + '?' + canonical_querystring + + # print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++') + # print('Request URL = ' + request_url) + try: + r = requests.post(request_url, headers=headers, data=req_body) + except Exception as err: + print(f'error occurred: {err}') + raise + else: + # print('\nRESPONSE++++++++++++++++++++++++++++++++++++') + # print(f'Response code: {r.status_code}\n') + # 使用 replace 方法将 \u0026 替换为 & + resp_str = r.text.replace("\\u0026", "&") + # print(f'Response body: {resp_str}\n') + return json.loads(resp_str) + +def round_to_multiple_of_16(x): + return int(round(x / 16)) * 16 + +def get_closest_api_size(orig_width, orig_height): + """ + 根据原始图片尺寸，返回最接近的API尺寸。 + """ + aspect = orig_width / orig_height + max_width, max_height = 2016, 1536 + max_aspect = max_width / max_height + + if aspect > max_aspect: + # 限制宽度 + width = max_width + height = width / aspect + else: + # 限制高度 + height = max_height + width = aspect * height + + # 四舍五入到16的倍数 + width = round_to_multiple_of_16(width) + height = round_to_multiple_of_16(height) + + # 再次确保合法范围 + width = min(max(width, 512), 2016) + height = min(max(height, 512), 1536) + + return width, height + + + +class LS_Jimeng_i2i_API: + + def __init__(self): + self.NODE_NAME = 'Jimeng Image2Image API' + pass + + @classmethod + def INPUT_TYPES(self): + return { + "required": { + "image": ("IMAGE",), + "model": (jimeng_i2i_model_list,), + "time_out": ("INT", {"default": 300, "min": 1, "max": 3600, "step": 1}), # 300s = 5min + "scale": ("FLOAT", {"default": 0.5, "min": 0, "max": 1, "step": 0.1}), + "seed": ("INT", {"default": 0, "min": 0, "max": 1e18, "step": 1}), + "prompt": ("STRING", {"default": "", "multiline": True}), + }, + "optional": { + } + } + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("image",) + FUNCTION = 'run_jimeng_i2i_api' + CATEGORY = '😺dzNodes/LayerUtility' + + def run_jimeng_i2i_api(self, image, model, time_out, scale, seed, prompt): + ret_images = [] + access_key = get_api_key("volcengine_AccessKeyId") + secret_key = get_api_key("volcengine_SecretAccessKey") + + # image shape = b, h, w, c + orig_width = image.shape[2] + orig_height = image.shape[1] + output_width, output_height = get_closest_api_size(orig_width, orig_height) + + for img in image: + img = torch.unsqueeze(img, 0) + orig_image = tensor2pil(img).convert('RGB') + orig_image = fit_resize_image(orig_image, output_width, output_height, 'fill', Image.BICUBIC) + buffered = io.BytesIO() + orig_image.save(buffered, format="JPEG") + image_bytes = buffered.getvalue() + image_base64 = base64.b64encode(image_bytes).decode("utf-8") + body_params = { + "req_key": model, + "binary_data_base64": [image_base64], + "prompt": prompt, + "seed": seed, + "scale": scale, + "width": output_width, + "height": output_height, + } + formatted_body = json.dumps(body_params) + + query_params = { + 'Action': 'CVSync2AsyncSubmitTask', + 'Version': '2022-08-31', + } + formatted_query = formatQuery(query_params) + + # 发送请求 + + result = signV4Request(access_key, secret_key, service, formatQuery(query_params), formatted_body) + response_code = result["code"] + if response_code > 10000: + print(f"Error: {result}") + raise Exception(f"error code: {response_code}") + task_id = result["data"]["task_id"] + log(f"Send Request to Jimeng API: task_id = {task_id}") + + # 获取结果 + start_time = time.time() + check_interval = 1 + check_timeout = 60 + + query_params["Action"] = 'CVSync2AsyncGetResult' + formatted_query = formatQuery(query_params) + + body_params = { + "req_key": "jimeng_i2i_v30", + "task_id": task_id, + } + formatted_body = json.dumps(body_params) + + while True: + time.sleep(check_interval) + result = signV4Request(access_key, secret_key, service, formatted_query, formatted_body) + query_status = result["data"]["status"] + if query_status == "done": + # print(f"query_status: {query_status}") + break + if time.time() - start_time > check_timeout: + raise Exception("Jimeng API Timeout") + + end_time = time.time() + use_time = round(end_time - start_time, 2) + log(f"Jimeng API responded in {use_time} seconds.") + + base64_str = result["data"]["binary_data_base64"][0] + image_data = base64.b64decode(base64_str) + ret_image = Image.open(io.BytesIO(image_data)) + ret_images.append(pil2tensor(ret_image)) + + return (torch.cat(ret_images, dim=0),) + + +NODE_CLASS_MAPPINGS = { + "LayerUtility: JimengI2IAPI": LS_Jimeng_i2i_API, +} + +NODE_DISPLAY_NAME_MAPPINGS = { + "LayerUtility: JimengI2IAPI": "LayerUtility: Jimeng Imgae to Image API (Advance)", +} \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 2ae4da0..641ec10 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "comfyui_layerstyle_advance" description = "The nodes detached from ComfyUI Layer Style are mainly those with complex requirements for dependency packages." -version = "2.0.22" +version = "2.0.23" license = { text = "MIT License" } dependencies = ["numpy", "matplotlib", "scikit_image", "scikit_learn", "opencv-contrib-python", "pymatting", "timm", "blend_modes", "transformers", "diffusers", "loguru", "colour-science", "huggingface_hub", "segment_anything", "addict", "omegaconf", "yapf", "wget", "iopath", "mediapipe", "typer_config", "fastapi", "rich", "google-generativeai", "ultralytics", "transparent-background", "accelerate", "onnxruntime", "bitsandbytes", "peft", "protobuf", "hydra-core", "blind-watermark", "qrcode", "pyzbar", "psd-tools", "wandb", "zhipuai", "openai","google-genai"]