Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
54a504e122 | ||
|
|
36876b888f | ||
|
|
af42216bd0 | ||
|
|
89f0ef3fe6 | ||
|
|
db0b8fa266 | ||
|
|
4092be6a0a | ||
|
|
1f493b180e | ||
|
|
fafef977a1 | ||
|
|
8715c032ae | ||
|
|
7da714c686 | ||
|
|
ee2debe867 | ||
|
|
c2d2ec96b2 | ||
|
|
20093d1c40 | ||
|
|
7a3091df10 | ||
|
|
bd7d00b1ce | ||
|
|
2be7d1e881 | ||
|
|
568f2b2e15 | ||
|
|
681f6ebc5f | ||
|
|
3492952986 | ||
|
|
7d350330e7 | ||
|
|
cde31dc9a7 | ||
|
|
cdc9e245ba | ||
|
|
9064c5b66e | ||
|
|
272f918f28 | ||
|
|
f3a5a3e6f6 |
@@ -3,3 +3,8 @@ __pycache__
|
||||
py/__pycache__
|
||||
py/modelverse_api/__pycache__
|
||||
py/modelverse_api/requests/__pycache__
|
||||
/venv
|
||||
.idea/
|
||||
.ace-tool/
|
||||
docs
|
||||
secrets.json
|
||||
@@ -1,59 +1,81 @@
|
||||
# UCloud ModelVerse API for ComfyUI
|
||||
|
||||
## Supported models
|
||||
This plugin provides seamless integration with UCloud's ModelVerse API, enabling you to use powerful AI models directly within ComfyUI for image generation and editing tasks.
|
||||
|
||||
Now we do support following models:
|
||||
## Quick Start
|
||||
|
||||
- Flux Dev
|
||||
- Flux Kontext Pro Text2Image
|
||||
- Flux Kontext Pro
|
||||
- Flux Kontext Pro (Multi-inputs)
|
||||
- Flux Kontext Max Text2Image
|
||||
- Flux Kontext Max
|
||||
- Flux Kontext Max (Multi-inputs)
|
||||
- Step1X-Edit
|
||||
### Installation
|
||||
|
||||
You can find the corresponding nodes in our node set.
|
||||
1. **Install via Custom Node Manager** (Recommended)
|
||||
- Open ComfyUI
|
||||
- Go to **Manager** → **Custom Nodes Manager**
|
||||
- Search for "ComfyUl-UCloud"
|
||||
- Click **Install**
|
||||
- Restart ComfyUI
|
||||
|
||||
Note: (Multi-inputs) uses same node as normal (single input). Please check out the example workflow hidden in the images below.
|
||||
2. **Configure API Key**
|
||||
|
||||
- Get your API key: [Console En](https://astraflow.ucloud.cn/modelverse/api-keys), [Console Zh](https://astraflow.ucloud.cn/modelverse/api-keys)
|
||||
- Add your UCloud ModelVerse API key in the `Modelverse Secret Client` node
|
||||
- Or Alternatively, create a `config.ini` file with `MODELVERSE_API_KEY=your_api_key`
|
||||
|
||||
## Instruction
|
||||
### Basic Usage
|
||||
|
||||
1. Install ComfyUI, please refer to [ComfyUI official github](https://github.com/comfyanonymous/ComfyUI).
|
||||
1. Add a `Modelverse Secret Client` node to your workflow
|
||||
2. Choose your desired model from the supported models below
|
||||
3. Connect the appropriate input/output nodes
|
||||
4. Run your workflow
|
||||
|
||||
Notice: this node plugin is only designed for ComfyUI version earlier than v0.3.44, please specify the release sersion when cloning. (e.g. `git clone -b v0.3.44 https://github.com/comfyanonymous/ComfyUI.git`)
|
||||
## Supported Models
|
||||
|
||||
2. Change to `ComfyUI/custom_nodes` directory, and clone this repo, install required python packages.
|
||||
Our plugin supports the following models, each optimized for specific use cases:
|
||||
|
||||
```
|
||||
cd ~/ComfyUI/custom_nodes
|
||||
git clone https://github.com/ucloud/ucloud-comfyui.git
|
||||
cd ./ucloud-comfyui
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
### Image Generation
|
||||
|
||||
3. Back to `ComfyUI` directory, start ComfyUI service, use web browser to open `localhost:8188`.
|
||||
- **Flux Dev** - Foundation model with strong stylization capabilities
|
||||
- **Flux Kontext Pro Text2Image** - Specialized for text-to-image generation
|
||||
- **Flux Kontext Pro** - Expert single image editing and enhancement
|
||||
- **Flux Kontext Pro (Multi-inputs)** - Batch processing for multiple images
|
||||
- **Flux Kontext Max Text2Image** - Premium text-to-image generation with highest quality
|
||||
- **Flux Kontext Max** - Flagship single image editing with advanced features
|
||||
- **Flux Kontext Max (Multi-inputs)** - Most powerful multi-image processing
|
||||
- **Step1X-Edit** - Precision local editing and inpainting
|
||||
- **GPT Image 1** - OpenAI's text-to-image generation
|
||||
- **Gemini 3 Pro Image** - Google's professional asset production with search grounding
|
||||
- **Qwen Image** - Alibaba's text-to-image generation
|
||||
|
||||
```
|
||||
cd ~/ComfyUI
|
||||
python main.py
|
||||
```
|
||||
### Video Generation
|
||||
|
||||
4. Build your own workflow with Modelverse nodes. You can check the following images, drag them into ComfyUI GUI and autoload example workflows. Don't forget to enter your `API_KEY` in the `Modelverse Client` node. If you wish to hide your `API_KEY` instead of showing it in GUI, alternatively you can write it in `config.ini` for `MODELVERSE_API_KEY` value.
|
||||
- **Kling V3** - Unified text/image-to-video and motion control
|
||||
- **Kling V3 Omni** - Multimodal video generation with reference inputs
|
||||
- **Doubao Seedance 2.0** - Text/image-to-video with audio generation
|
||||
- **OpenAI Sora 2** - High-quality text-to-video generation
|
||||
- **HappyHorse 1.0** - Text-to-video generation
|
||||
- **Veo 3.1** - Google's video generation (text-to-video, image-to-video)
|
||||
- **Vidu** - Multiple models for text/image-to-video (viduq3-pro, viduq3-turbo, viduq2)
|
||||
- **Wan-AI** - Text/image-to-video generation
|
||||
|
||||
| Flux Dev | Flux Kontext Pro | Flux Kontext Max (Multi-inputs) |
|
||||
|:-:|:-:|:-:|
|
||||
| <img src="assets/flux_dev.png" width="200" height="200"> | <img src="assets/flux_kontext_pro_single.png" width="200" height="200"> | <img src="assets/flux_kontext_max_multi.png" width="200" height="200"> |
|
||||
Note: (Multi-inputs) models use the same node interface as their single-input counterparts. Check the example workflows below for implementation details.
|
||||
|
||||
In case the above images does not work for you, check the screenshot below:
|
||||
## Example Workflows
|
||||
|
||||
Text-to-Image with Flux Dev
|
||||

|
||||
Build your own workflow with Modelverse nodes. You can check the following images, drag them into ComfyUI GUI and autoload example workflows.
|
||||
|
||||
| Flux Dev | Flux Kontext Pro | Flux Kontext Max (Multi-inputs) |
|
||||
|:-:|:-:|:-:|
|
||||
| <img src="assets/flux_dev.png" width="200" height="200"> | <img src="assets/flux_kontext_pro_single.png" width="200" height="200"> | <img src="assets/flux_kontext_max_multi.png" width="200" height="200"> |
|
||||
|
||||
In case the above images does not work for you, check the screenshot below:
|
||||
|
||||
Text-to-Image with Flux Dev
|
||||

|
||||
|
||||
Single image editting with Flux Kontext Pro
|
||||

|
||||
|
||||
Multi image editting with Flux Kontext Max
|
||||

|
||||
|
||||
Single image editting with Flux Kontext Pro
|
||||

|
||||
|
||||
Multi image editting with Flux Kontext Max
|
||||

|
||||
|
||||
|
||||
+2
-1
@@ -39,7 +39,8 @@ for file in files:
|
||||
serialized_CLASS_MAPPINGS = {k: serialize(v) for k, v in imported_module.NODE_CLASS_MAPPINGS.items()}
|
||||
serialized_DISPLAY_NAME_MAPPINGS = {k: serialize(v) for k, v in imported_module.NODE_DISPLAY_NAME_MAPPINGS.items()}
|
||||
all_nodes[file]={"NODE_CLASS_MAPPINGS": serialized_CLASS_MAPPINGS, "NODE_DISPLAY_NAME_MAPPINGS": serialized_DISPLAY_NAME_MAPPINGS}
|
||||
except:
|
||||
except Exception as e:
|
||||
print(f"Failed to import {file}: {e}")
|
||||
pass
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""
|
||||
Doubao Seedance 2.0 - Text/image-to-video model
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.doubao_seedance_2 import (
|
||||
DoubaoSeedance2,
|
||||
MODEL,
|
||||
RESOLUTIONS,
|
||||
RATIOS,
|
||||
)
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class DoubaoSeedance2Node:
|
||||
"""
|
||||
Doubao Seedance 2.0 text/image-to-video generation.
|
||||
Supports text, first/last frame images, reference image/video/audio inputs.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video (recommended: up to 500 characters)",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image"}),
|
||||
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "First frame image URL (use either this OR first_frame_image, not both)"}),
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image"}),
|
||||
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "Last frame image URL (use either this OR last_frame_image, not both)"}),
|
||||
"reference_image": (IO.IMAGE, {"tooltip": "Reference image"}),
|
||||
"reference_image_url": (IO.STRING, {"default": "", "tooltip": "Reference image URL (use either this OR reference_image, not both)"}),
|
||||
"reference_video_url": (IO.STRING, {"default": "", "tooltip": "Reference video URL"}),
|
||||
"reference_audio_url": (IO.STRING, {"default": "", "tooltip": "Reference audio URL"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 4, "max": 15, "step": 1, "tooltip": "Video duration in seconds (4–15)"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "Output resolution; draft mode supports 480p only"}),
|
||||
"ratio": (RATIOS, {"default": "adaptive", "tooltip": "Aspect ratio; adaptive auto-selects the best fit"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "Random seed for reproducible results"}),
|
||||
"generate_audio": (IO.BOOLEAN, {"default": False, "tooltip": "Generate audio synchronized with the video"}),
|
||||
"camera_fixed": (IO.BOOLEAN, {"default": False, "tooltip": "Fix camera position (no camera movement)"}),
|
||||
"watermark": (IO.BOOLEAN, {"default": False, "tooltip": "Add watermark to the output video"}),
|
||||
"draft": (IO.BOOLEAN, {"default": False, "tooltip": "Draft/preview mode (480p only)"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Seedance"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
prompt,
|
||||
first_frame_image=None,
|
||||
first_frame_url="",
|
||||
last_frame_image=None,
|
||||
last_frame_url="",
|
||||
reference_image=None,
|
||||
reference_image_url="",
|
||||
reference_video_url="",
|
||||
reference_audio_url="",
|
||||
duration=5,
|
||||
resolution="720p",
|
||||
ratio="adaptive",
|
||||
seed=0,
|
||||
generate_audio=False,
|
||||
camera_fixed=False,
|
||||
watermark=False,
|
||||
draft=False,
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
request = DoubaoSeedance2(
|
||||
prompt=prompt,
|
||||
first_frame=first_frame_image,
|
||||
first_frame_url=first_frame_url,
|
||||
last_frame=last_frame_image,
|
||||
last_frame_url=last_frame_url,
|
||||
reference_image=reference_image,
|
||||
reference_image_url=reference_image_url,
|
||||
reference_video_url=reference_video_url,
|
||||
reference_audio_url=reference_audio_url,
|
||||
duration=duration,
|
||||
resolution=resolution,
|
||||
ratio=ratio,
|
||||
seed=seed,
|
||||
generate_audio=generate_audio,
|
||||
camera_fixed=camera_fixed,
|
||||
watermark=watermark,
|
||||
draft=draft,
|
||||
)
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
print(f"Submitting Seedance 2.0 task: model={MODEL}, prompt={prompt!r}")
|
||||
submit_res = mv_client.submit_task_request(request)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Seedance 2.0 task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Doubao_Seedance_2": DoubaoSeedance2Node,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Doubao_Seedance_2": "Modelverse Doubao Seedance 2.0",
|
||||
}
|
||||
+12
-12
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.flux_dev import FluxDev
|
||||
import torch
|
||||
import asyncio
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class FluxDevNode:
|
||||
@@ -17,8 +17,8 @@ class FluxDevNode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"width": ("INT", {
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"width": (IO.INT, {
|
||||
"default": 1024,
|
||||
"min": 512,
|
||||
"max": 1536,
|
||||
@@ -26,7 +26,7 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Image width (512 to 1536)"
|
||||
}),
|
||||
"height": ("INT", {
|
||||
"height": (IO.INT, {
|
||||
"default": 1024,
|
||||
"min": 512,
|
||||
"max": 1536,
|
||||
@@ -34,7 +34,7 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Image height (512 to 1536)"
|
||||
}),
|
||||
"strength": ("FLOAT", {
|
||||
"strength": (IO.FLOAT, {
|
||||
"default": 0.8,
|
||||
"min": 0.01,
|
||||
"max": 1.0,
|
||||
@@ -42,14 +42,14 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Strength of the image-to-image transformation (0.01 to 1.0)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"num_images": ("INT", {
|
||||
"num_images": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 4,
|
||||
@@ -57,7 +57,7 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of images to generate in a single request (1 to 4)"
|
||||
}),
|
||||
"num_requests": ("INT", {
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -65,7 +65,7 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"num_inference_steps": ("INT", {
|
||||
"num_inference_steps": (IO.INT, {
|
||||
"default": 28,
|
||||
"min": 1,
|
||||
"max": 50,
|
||||
@@ -73,7 +73,7 @@ class FluxDevNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of inference steps (1 to 50)"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 3.5,
|
||||
"min": 0.0,
|
||||
"max": 10.0,
|
||||
@@ -83,7 +83,7 @@ class FluxDevNode:
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"image": ("IMAGE", {
|
||||
"image": (IO.IMAGE, {
|
||||
"tooltip": "The image for reference.",
|
||||
"forceInput": False,
|
||||
"default": None
|
||||
@@ -94,7 +94,7 @@ class FluxDevNode:
|
||||
RETURN_TYPES = ("IMAGE", )
|
||||
RETURN_NAMES = ("image",)
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Flux"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(self,
|
||||
|
||||
+16
-9
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.flux_kontext_max import FluxKontextMax, FluxKontextMaxMulti
|
||||
import torch
|
||||
import asyncio
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class FluxKontextMaxNode:
|
||||
@@ -21,13 +21,13 @@ class FluxKontextMaxNode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"images": ("IMAGE,IMAGE_LIST", {
|
||||
"tooltip": "The image(s) to edit from. If not included, use text-to-image mode.",
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"images": (IO.IMAGE, {
|
||||
"tooltip": "Image(s) to edit. Connect a single IMAGE, or use Modelverse Image Packer for multiple images.",
|
||||
"forceInput": False,
|
||||
"default": None
|
||||
}),
|
||||
"num_requests": ("INT", {
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -35,14 +35,14 @@ class FluxKontextMaxNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 2.5,
|
||||
"min": 0.0,
|
||||
"max": 10.0,
|
||||
@@ -74,9 +74,16 @@ class FluxKontextMaxNode:
|
||||
|
||||
mode = "single"
|
||||
if isinstance(images, list):
|
||||
if len(images) > 1:
|
||||
print("INFO:", "Running Flux Kontext Max multi-image edit mode.")
|
||||
print("INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
mode = "multi"
|
||||
else:
|
||||
images = images[0]
|
||||
elif isinstance(images, torch.Tensor) and images.ndim == 4 and images.shape[0] > 1:
|
||||
images = [images[i:i + 1] for i in range(images.shape[0])]
|
||||
print("INFO:", "Running Flux Kontext Max multi-image edit mode.")
|
||||
print(
|
||||
"INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
print("INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
mode = "multi"
|
||||
else:
|
||||
print("INFO:", "Running Flux Kontext Max single-image edit mode.")
|
||||
|
||||
@@ -2,8 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.flux_kontext_max import FluxKontextMaxT2I
|
||||
import torch
|
||||
import asyncio
|
||||
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
class FluxKontextMaxT2INode:
|
||||
"""
|
||||
@@ -18,12 +17,12 @@ class FluxKontextMaxT2INode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"aspect_ratio": (["21:9", "16:9", "16:10", "4:3", "1:1", "3:4", "10:16", "9:16", "9:21"], {
|
||||
"default": "1:1",
|
||||
"tooltip": "The aspect ratio of the output image, ranging from \"21:9\" to \"9:21\", default is \"1:1\""
|
||||
}),
|
||||
"num_images": ("INT", {
|
||||
"num_images": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 4,
|
||||
@@ -31,7 +30,7 @@ class FluxKontextMaxT2INode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of images to generate in a single request (1 to 4)"
|
||||
}),
|
||||
"num_requests": ("INT", {
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -39,14 +38,14 @@ class FluxKontextMaxT2INode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 2.5,
|
||||
"min": 0.0,
|
||||
"max": 10.0,
|
||||
@@ -60,7 +59,7 @@ class FluxKontextMaxT2INode:
|
||||
RETURN_TYPES = ("IMAGE", )
|
||||
RETURN_NAMES = ("image",)
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Flux"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(self,
|
||||
|
||||
+17
-10
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.flux_kontext_pro import FluxKontextPro, FluxKontextProMulti
|
||||
import torch
|
||||
import asyncio
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class FluxKontextProNode:
|
||||
@@ -21,13 +21,13 @@ class FluxKontextProNode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"images": ("IMAGE,IMAGE_LIST", {
|
||||
"tooltip": "The image(s) to edit from. If not included, use text-to-image mode.",
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"images": (IO.IMAGE, {
|
||||
"tooltip": "Image(s) to edit. Connect a single IMAGE, or use Modelverse Image Packer for multiple images.",
|
||||
"forceInput": False,
|
||||
"default": None
|
||||
}),
|
||||
"num_requests": ("INT", {
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -35,14 +35,14 @@ class FluxKontextProNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 2.5,
|
||||
"min": 0.0,
|
||||
"max": 10.0,
|
||||
@@ -56,7 +56,7 @@ class FluxKontextProNode:
|
||||
RETURN_TYPES = ("IMAGE", )
|
||||
RETURN_NAMES = ("image",)
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Flux"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(self,
|
||||
@@ -74,9 +74,16 @@ class FluxKontextProNode:
|
||||
|
||||
mode = "single"
|
||||
if isinstance(images, list):
|
||||
if len(images) > 1:
|
||||
print("INFO:", "Running Flux Kontext Pro multi-image edit mode.")
|
||||
print("INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
mode = "multi"
|
||||
else:
|
||||
images = images[0]
|
||||
elif isinstance(images, torch.Tensor) and images.ndim == 4 and images.shape[0] > 1:
|
||||
images = [images[i:i + 1] for i in range(images.shape[0])]
|
||||
print("INFO:", "Running Flux Kontext Pro multi-image edit mode.")
|
||||
print(
|
||||
"INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
print("INFO:", f"{len(images)} image included for the multi-image edit.")
|
||||
mode = "multi"
|
||||
else:
|
||||
print("INFO:", "Running Flux Kontext Pro single-image edit mode.")
|
||||
|
||||
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.flux_kontext_pro import FluxKontextProT2I
|
||||
import torch
|
||||
import asyncio
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class FluxKontextProT2INode:
|
||||
@@ -18,12 +18,12 @@ class FluxKontextProT2INode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
|
||||
"aspect_ratio": (["21:9", "16:9", "16:10", "4:3", "1:1", "3:4", "10:16", "9:16", "9:21"], {
|
||||
"default": "1:1",
|
||||
"tooltip": "The aspect ratio of the output image, ranging from \"21:9\" to \"9:21\", default is \"1:1\""
|
||||
}),
|
||||
"num_images": ("INT", {
|
||||
"num_images": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 4,
|
||||
@@ -31,7 +31,7 @@ class FluxKontextProT2INode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of images to generate in a single request (1 to 4)"
|
||||
}),
|
||||
"num_requests": ("INT", {
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -39,14 +39,14 @@ class FluxKontextProT2INode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 2.5,
|
||||
"min": 0.0,
|
||||
"max": 10.0,
|
||||
@@ -60,7 +60,7 @@ class FluxKontextProT2INode:
|
||||
RETURN_TYPES = ("IMAGE", )
|
||||
RETURN_NAMES = ("image",)
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Flux"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(self,
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
import base64
|
||||
import torch
|
||||
from typing import Optional, List, Dict, Any
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.gemini_flash_image import GeminiFlashImageRequest
|
||||
from .modelverse_api.utils import decode_image, images2tensor
|
||||
|
||||
|
||||
MODELS = ["gemini-3.1-flash-image", "gemini-2.5-flash-image"]
|
||||
|
||||
|
||||
def _extract_images_from_gemini_response(resp: Dict[str, Any]) -> List[torch.Tensor]:
|
||||
"""Parse Gemini API response and return list of tensors for any inline images."""
|
||||
images = []
|
||||
try:
|
||||
candidates = resp.get("candidates", []) or []
|
||||
for cand in candidates:
|
||||
content = cand.get("content", {})
|
||||
parts = content.get("parts", [])
|
||||
for part in parts:
|
||||
inline = part.get("inlineData")
|
||||
if inline and isinstance(inline, dict):
|
||||
data_b64 = inline.get("data")
|
||||
if data_b64:
|
||||
try:
|
||||
img_bytes = base64.b64decode(data_b64)
|
||||
pil_img = decode_image(img_bytes)
|
||||
images.append(pil_img)
|
||||
except Exception:
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
if not images:
|
||||
return []
|
||||
return [images2tensor(images)]
|
||||
|
||||
|
||||
class GeminiFlashImageNode:
|
||||
"""
|
||||
Gemini Flash Image (text-to-image and image-edit) via Modelverse API.
|
||||
|
||||
Endpoint: /v1beta/models/{model}:generateContent
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "gemini-3.1-flash-image", "tooltip": "Gemini Flash Image model"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme"}),
|
||||
"mime_type": (["image/png", "image/jpeg"], {"default": "image/png"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
},
|
||||
"optional": {
|
||||
"image": (IO.IMAGE, {"default": None, "tooltip": "Optional input image for edit"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Gemini"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(self,
|
||||
client,
|
||||
model: str,
|
||||
prompt: str,
|
||||
mime_type: str = "image/png",
|
||||
num_requests: int = 1,
|
||||
image=None):
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
outputs: List[torch.Tensor] = []
|
||||
for i in range(num_requests):
|
||||
req = GeminiFlashImageRequest(prompt=prompt, model=model, image=image, mime_type=mime_type)
|
||||
payload = req.build_payload()
|
||||
resp = mv_client.post(req.API_PATH, payload)
|
||||
|
||||
if isinstance(resp, dict) and resp.get("error"):
|
||||
err = resp.get("error")
|
||||
raise Exception(f"GeminiFlashImage error: {err.get('message', 'Unknown error')}")
|
||||
|
||||
tensors = _extract_images_from_gemini_response(resp)
|
||||
if not tensors:
|
||||
print("WARN:", "No image data found in Gemini response; check console for details.")
|
||||
continue
|
||||
# tensors is a list with one batch tensor; append that tensor
|
||||
outputs.append(tensors[0])
|
||||
|
||||
if not outputs:
|
||||
# Return an empty black image tensor (1x3x1x1) to avoid breaking graph
|
||||
return (torch.zeros((1, 3, 1, 1)),)
|
||||
|
||||
# Concatenate along batch dimension
|
||||
return (torch.cat(outputs, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"NanoBanana": GeminiFlashImageNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"NanoBanana": "Modelverse Gemini Flash Image",
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
"""
|
||||
Gemini 3 Pro Image (Nano Banana Pro) ComfyUI Node.
|
||||
Professional asset production with advanced features:
|
||||
- High-resolution output (1K, 2K, 4K)
|
||||
- Aspect ratio control
|
||||
- Google Search grounding
|
||||
- Up to 14 reference images
|
||||
"""
|
||||
import base64
|
||||
import torch
|
||||
from typing import Optional, List, Dict, Any
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.gemini_pro_image import GeminiProImageRequest
|
||||
from .modelverse_api.utils import decode_image, images2tensor
|
||||
|
||||
|
||||
def _extract_images_from_gemini_response(resp: Dict[str, Any]) -> List[torch.Tensor]:
|
||||
"""Extract image tensors from Gemini API response."""
|
||||
tensors: List[torch.Tensor] = []
|
||||
candidates = resp.get("candidates", [])
|
||||
for cand in candidates:
|
||||
content = cand.get("content", {})
|
||||
parts = content.get("parts", [])
|
||||
for part in parts:
|
||||
inline = part.get("inlineData")
|
||||
if inline:
|
||||
data_b64 = inline.get("data")
|
||||
if data_b64:
|
||||
raw = base64.b64decode(data_b64)
|
||||
pil_img = decode_image(raw)
|
||||
tensors.append(images2tensor(pil_img))
|
||||
# Also print any text response
|
||||
if part.get("text"):
|
||||
print("INFO: Gemini Pro Image text response:", part.get("text"))
|
||||
return tensors
|
||||
|
||||
|
||||
# Aspect ratio options
|
||||
ASPECT_RATIOS = ["auto", "1:1", "2:3", "3:2", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"]
|
||||
# Resolution options
|
||||
IMAGE_SIZES = ["1K", "2K", "4K"]
|
||||
|
||||
|
||||
class GeminiProImageNode:
|
||||
"""
|
||||
Gemini 3 Pro Image (Nano Banana Pro) via Modelverse API.
|
||||
|
||||
Professional asset production with:
|
||||
- High-resolution output (1K, 2K, 4K)
|
||||
- Aspect ratio control
|
||||
- Google Search grounding for real-time info
|
||||
- Up to 14 reference images
|
||||
|
||||
Endpoint: /v1beta/models/gemini-3-pro-image:generateContent
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "Create a professional product photo"}),
|
||||
"mime_type": (["image/png", "image/jpeg"], {"default": "image/png"}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {"default": "auto"}),
|
||||
"image_size": (IMAGE_SIZES, {"default": "1K"}),
|
||||
"use_google_search": (IO.BOOLEAN, {"default": False, "tooltip": "Enable Google Search grounding for real-time info"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
},
|
||||
"optional": {
|
||||
"image1": (IO.IMAGE, {"default": None, "tooltip": "Reference image 1"}),
|
||||
"image2": (IO.IMAGE, {"default": None, "tooltip": "Reference image 2"}),
|
||||
"image3": (IO.IMAGE, {"default": None, "tooltip": "Reference image 3"}),
|
||||
"image4": (IO.IMAGE, {"default": None, "tooltip": "Reference image 4"}),
|
||||
"image5": (IO.IMAGE, {"default": None, "tooltip": "Reference image 5"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Gemini"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(
|
||||
self,
|
||||
client,
|
||||
prompt: str,
|
||||
mime_type: str = "image/png",
|
||||
aspect_ratio: str = "auto",
|
||||
image_size: str = "1K",
|
||||
use_google_search: bool = False,
|
||||
num_requests: int = 1,
|
||||
image1=None,
|
||||
image2=None,
|
||||
image3=None,
|
||||
image4=None,
|
||||
image5=None,
|
||||
):
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
# Collect input images
|
||||
images = []
|
||||
for img in [image1, image2, image3, image4, image5]:
|
||||
if img is not None:
|
||||
images.append(img)
|
||||
|
||||
# Process aspect_ratio
|
||||
ar = aspect_ratio if aspect_ratio != "auto" else None
|
||||
|
||||
outputs: List[torch.Tensor] = []
|
||||
for i in range(num_requests):
|
||||
req = GeminiProImageRequest(
|
||||
prompt=prompt,
|
||||
images=images if images else None,
|
||||
mime_type=mime_type,
|
||||
aspect_ratio=ar,
|
||||
image_size=image_size,
|
||||
use_google_search=use_google_search,
|
||||
)
|
||||
payload = req.build_payload()
|
||||
resp = mv_client.post(req.API_PATH, payload)
|
||||
|
||||
if isinstance(resp, dict) and resp.get("error"):
|
||||
err = resp.get("error")
|
||||
raise Exception(f"GeminiProImage error: {err.get('message', 'Unknown error')}")
|
||||
|
||||
tensors = _extract_images_from_gemini_response(resp)
|
||||
if not tensors:
|
||||
print("WARN:", "No image data found in Gemini Pro response; check console for details.")
|
||||
continue
|
||||
outputs.append(tensors[0])
|
||||
|
||||
if not outputs:
|
||||
raise Exception("No images generated from Gemini 3 Pro Image")
|
||||
|
||||
return (torch.cat(outputs, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
'Gemini Pro Image (Nano Banana Pro)': GeminiProImageNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
'Gemini Pro Image (Nano Banana Pro)': 'Gemini 3 Pro Image',
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
import base64
|
||||
import torch
|
||||
from typing import List
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.gpt_image_1 import GPTImage1
|
||||
|
||||
|
||||
class GPTImage1Node:
|
||||
"""
|
||||
gpt-image-1 text-to-image via Modelverse /v1/images/generations API.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower"}),
|
||||
"size": (["1024x1024", "1024x1536", "1536x1024"], {"default": "1024x1024"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
|
||||
},
|
||||
"optional": {
|
||||
"quality": (["", "low", "medium", "high"], {"default": ""}),
|
||||
"output_format": (["png", "jpeg"], {"default": "png"}),
|
||||
"output_compression": (IO.INT, {"default": 100, "min": 0, "max": 100, "step": 1, "display": "number"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Gpt-img"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(
|
||||
self,
|
||||
client,
|
||||
prompt: str,
|
||||
size: str = "1024x1024",
|
||||
num_requests: int = 1,
|
||||
num_images: int = 1,
|
||||
quality: str = "",
|
||||
output_format: str = "png",
|
||||
output_compression: int = 100,
|
||||
):
|
||||
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
tasks = [
|
||||
mv_client.async_send_request(
|
||||
GPTImage1(
|
||||
prompt=prompt,
|
||||
num_images=num_images,
|
||||
size=size,
|
||||
quality=quality if quality != "" else None,
|
||||
output_format=output_format,
|
||||
output_compression=output_compression,
|
||||
)
|
||||
)
|
||||
for i in range(num_requests)
|
||||
]
|
||||
|
||||
results = await mv_client.run_tasks(tasks)
|
||||
|
||||
output_images_list: List[torch.Tensor] = []
|
||||
for data_list in results:
|
||||
if not data_list:
|
||||
print("WARN:", "No output in current request. Skipping...")
|
||||
continue
|
||||
|
||||
# Auto-detect output type: prefer URL if present, else decode b64_json
|
||||
has_url = False
|
||||
try:
|
||||
for item in data_list:
|
||||
if isinstance(item, dict) and item.get("url"):
|
||||
has_url = True
|
||||
break
|
||||
except Exception:
|
||||
has_url = False
|
||||
|
||||
if has_url:
|
||||
try:
|
||||
output_images = imageurl2tensor(data_list)
|
||||
except Exception:
|
||||
print("WARN:", "Failed to load URL images; attempting b64_json decode.")
|
||||
has_url = False # fall through to b64 decode
|
||||
|
||||
if not has_url:
|
||||
images = []
|
||||
for item in data_list:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
b64v = item.get("b64_json") or item.get("b64")
|
||||
if not b64v:
|
||||
continue
|
||||
if isinstance(b64v, str) and b64v.startswith("data:"):
|
||||
try:
|
||||
b64v = b64v.split(",", 1)[1]
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
img_bytes = base64.b64decode(b64v)
|
||||
pil_img = decode_image(img_bytes)
|
||||
images.append(pil_img)
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
if not images:
|
||||
print("WARN:", "No decodable base64 image found.")
|
||||
continue
|
||||
output_images = images2tensor(images)
|
||||
|
||||
output_images_list.append(output_images)
|
||||
|
||||
if not output_images_list:
|
||||
return (torch.zeros((1, 3, 1, 1)),)
|
||||
|
||||
return (torch.cat(output_images_list, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse GPTImage1Node": GPTImage1Node,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse GPTImage1Node": "Modelverse GPT Image 1",
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
import base64
|
||||
import torch
|
||||
from typing import List
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.gpt_image_1_edit import GPTImage1Edit
|
||||
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
|
||||
|
||||
|
||||
class GPTImage1EditNode:
|
||||
"""
|
||||
gpt-image-1 image edit via /v1/images/edits (multipart/form-data).
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "Add a beach ball in the center"}),
|
||||
"image": (IO.IMAGE, {"tooltip": "Input image"}),
|
||||
"size": (["1024x1024", "1024x1536", "1536x1024"], {"default": "1024x1024"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
|
||||
},
|
||||
"optional": {
|
||||
"mask": (IO.IMAGE, {"default": None, "tooltip": "Optional mask image"}),
|
||||
"quality": (["", "low", "medium", "high"], {"default": ""}),
|
||||
"output_format": (["png", "jpeg"], {"default": "png"}),
|
||||
"output_compression": (IO.INT, {"default": 100, "min": 0, "max": 100, "step": 1, "display": "number"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Gpt-img"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(
|
||||
self,
|
||||
client,
|
||||
prompt: str,
|
||||
image,
|
||||
size: str = "1024x1024",
|
||||
num_requests: int = 1,
|
||||
num_images: int = 1,
|
||||
mask=None,
|
||||
quality: str = "",
|
||||
output_format: str = "png",
|
||||
output_compression: int = 100,
|
||||
):
|
||||
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
if image is None:
|
||||
raise ValueError("Input image is required")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
tasks = [
|
||||
mv_client.async_send_request(
|
||||
GPTImage1Edit(
|
||||
prompt=prompt,
|
||||
image=image,
|
||||
mask=mask,
|
||||
num_images=num_images,
|
||||
size=size,
|
||||
quality=quality if quality != "" else None,
|
||||
output_format=output_format,
|
||||
output_compression=output_compression,
|
||||
)
|
||||
)
|
||||
for _ in range(num_requests)
|
||||
]
|
||||
|
||||
results = await mv_client.run_tasks(tasks)
|
||||
|
||||
output_images_list: List[torch.Tensor] = []
|
||||
for data_list in results:
|
||||
if not data_list:
|
||||
print("WARN:", "No output in current request. Skipping...")
|
||||
continue
|
||||
|
||||
# Auto-detect URL vs b64_json
|
||||
has_url = any(isinstance(it, dict) and it.get("url") for it in data_list)
|
||||
if has_url:
|
||||
try:
|
||||
output_images = imageurl2tensor(data_list)
|
||||
except Exception:
|
||||
print("WARN:", "Failed to load URL images; attempting b64_json decode.")
|
||||
has_url = False
|
||||
|
||||
if not has_url:
|
||||
images = []
|
||||
for item in data_list:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
b64v = item.get("b64_json") or item.get("b64")
|
||||
if not b64v:
|
||||
continue
|
||||
if isinstance(b64v, str) and b64v.startswith("data:"):
|
||||
try:
|
||||
b64v = b64v.split(",", 1)[1]
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
img_bytes = base64.b64decode(b64v)
|
||||
pil_img = decode_image(img_bytes)
|
||||
images.append(pil_img)
|
||||
except Exception:
|
||||
continue
|
||||
if not images:
|
||||
print("WARN:", "No decodable base64 image found.")
|
||||
continue
|
||||
output_images = images2tensor(images)
|
||||
|
||||
output_images_list.append(output_images)
|
||||
|
||||
if not output_images_list:
|
||||
return (torch.zeros((1, 3, 1, 1)),)
|
||||
|
||||
return (torch.cat(output_images_list, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse GPTImage1EditNode": GPTImage1EditNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse GPTImage1EditNode": "Modelverse GPT Image 1 Edit",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
"""
|
||||
HappyHorse 1.0 Image2Video
|
||||
Model: happyhorse-1.0-i2v
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODEL = "happyhorse-1.0-i2v"
|
||||
RESOLUTIONS = ["480P", "720P", "1080P"]
|
||||
|
||||
|
||||
class HappyHorseImg2VideoNode:
|
||||
"""
|
||||
HappyHorse 1.0 Image2Video - image-to-video generation.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image"}),
|
||||
"first_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "First frame image URL or base64 (use either this OR first_frame_image)",
|
||||
}),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Optional text prompt to guide video generation",
|
||||
}),
|
||||
"resolution": (RESOLUTIONS, {
|
||||
"default": "720P",
|
||||
"tooltip": "Output resolution tier",
|
||||
}),
|
||||
"duration": (IO.INT, {
|
||||
"default": 5, "min": 1, "max": 15, "step": 1,
|
||||
"tooltip": "Video duration in seconds",
|
||||
}),
|
||||
"seed": (IO.INT, {
|
||||
"default": 0, "min": 0, "max": 2147483647,
|
||||
"tooltip": "Random seed (0 for random)",
|
||||
}),
|
||||
"watermark": (IO.BOOLEAN, {
|
||||
"default": False,
|
||||
"tooltip": "Add watermark to output video",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
first_frame_image=None,
|
||||
first_frame_url="",
|
||||
prompt="",
|
||||
resolution="720P",
|
||||
duration=5,
|
||||
seed=0,
|
||||
watermark=False,
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
has_url = first_frame_url and first_frame_url.strip()
|
||||
has_image = first_frame_image is not None
|
||||
if has_url and has_image:
|
||||
raise ValueError("Provide either first_frame_image or first_frame_url, not both")
|
||||
if not has_url and not has_image:
|
||||
raise ValueError("first_frame_image or first_frame_url is required for HappyHorse I2V")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
task_input = {}
|
||||
if has_url:
|
||||
task_input["first_frame_url"] = first_frame_url.strip()
|
||||
else:
|
||||
task_input["first_frame_url"] = image_to_base64(first_frame_image)
|
||||
|
||||
if prompt and prompt.strip():
|
||||
task_input["prompt"] = prompt.strip()
|
||||
|
||||
parameters = {
|
||||
"resolution": resolution,
|
||||
"duration": duration,
|
||||
"seed": seed,
|
||||
}
|
||||
if watermark:
|
||||
parameters["watermark"] = True
|
||||
|
||||
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"HappyHorse I2V task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"HappyHorse_Img2Video": HappyHorseImg2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"HappyHorse_Img2Video": "Modelverse HappyHorse I2V",
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
"""
|
||||
HappyHorse 1.0 Reference2Video
|
||||
Model: happyhorse-1.0-r2v
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODEL = "happyhorse-1.0-r2v"
|
||||
RESOLUTIONS = ["480P", "720P", "1080P"]
|
||||
RATIOS = ["16:9", "9:16", "1:1", "3:4", "4:3"]
|
||||
|
||||
|
||||
class HappyHorseReference2VideoNode:
|
||||
"""
|
||||
HappyHorse 1.0 Reference2Video - reference-image-to-video generation.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"image1": (IO.IMAGE, {"tooltip": "Reference image 1"}),
|
||||
"image2": (IO.IMAGE, {"tooltip": "Reference image 2"}),
|
||||
"image3": (IO.IMAGE, {"tooltip": "Reference image 3"}),
|
||||
"image4": (IO.IMAGE, {"tooltip": "Reference image 4"}),
|
||||
"image_urls": (IO.STRING, {
|
||||
"default": "",
|
||||
"multiline": True,
|
||||
"tooltip": "Reference image URLs, one per line",
|
||||
}),
|
||||
"resolution": (RESOLUTIONS, {
|
||||
"default": "720P",
|
||||
"tooltip": "Output resolution tier",
|
||||
}),
|
||||
"ratio": (RATIOS, {
|
||||
"default": "16:9",
|
||||
"tooltip": "Aspect ratio",
|
||||
}),
|
||||
"duration": (IO.INT, {
|
||||
"default": 5, "min": 1, "max": 15, "step": 1,
|
||||
"tooltip": "Video duration in seconds",
|
||||
}),
|
||||
"seed": (IO.INT, {
|
||||
"default": 0, "min": 0, "max": 2147483647,
|
||||
"tooltip": "Random seed (0 for random)",
|
||||
}),
|
||||
"watermark": (IO.BOOLEAN, {
|
||||
"default": False,
|
||||
"tooltip": "Add watermark to output video",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
prompt,
|
||||
image1=None,
|
||||
image2=None,
|
||||
image3=None,
|
||||
image4=None,
|
||||
image_urls="",
|
||||
resolution="720P",
|
||||
ratio="16:9",
|
||||
duration=5,
|
||||
seed=0,
|
||||
watermark=False,
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("prompt is required for HappyHorse R2V")
|
||||
|
||||
images = []
|
||||
for img in [image1, image2, image3, image4]:
|
||||
if img is not None:
|
||||
images.append(image_to_base64(img))
|
||||
|
||||
if image_urls and image_urls.strip():
|
||||
for url in image_urls.strip().split("\n"):
|
||||
url = url.strip()
|
||||
if url:
|
||||
images.append(url)
|
||||
|
||||
if not images:
|
||||
raise ValueError("At least one reference image is required for HappyHorse R2V")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
task_input = {
|
||||
"prompt": prompt.strip(),
|
||||
"images": images,
|
||||
}
|
||||
parameters = {
|
||||
"resolution": resolution,
|
||||
"ratio": ratio,
|
||||
"duration": duration,
|
||||
"seed": seed,
|
||||
}
|
||||
if watermark:
|
||||
parameters["watermark"] = True
|
||||
|
||||
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"HappyHorse R2V task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"HappyHorse_Reference2Video": HappyHorseReference2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"HappyHorse_Reference2Video": "Modelverse HappyHorse R2V",
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
HappyHorse 1.0 Text2Video
|
||||
Model: happyhorse-1.0-t2v
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODEL = "happyhorse-1.0-t2v"
|
||||
RESOLUTIONS = ["480P", "720P", "1080P"]
|
||||
RATIOS = ["16:9", "9:16", "1:1", "3:4", "4:3"]
|
||||
|
||||
|
||||
class HappyHorseText2VideoNode:
|
||||
"""
|
||||
HappyHorse 1.0 Text2Video - text-to-video generation.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"resolution": (RESOLUTIONS, {
|
||||
"default": "720P",
|
||||
"tooltip": "Output resolution tier",
|
||||
}),
|
||||
"ratio": (RATIOS, {
|
||||
"default": "16:9",
|
||||
"tooltip": "Aspect ratio",
|
||||
}),
|
||||
"duration": (IO.INT, {
|
||||
"default": 5, "min": 1, "max": 15, "step": 1,
|
||||
"tooltip": "Video duration in seconds",
|
||||
}),
|
||||
"seed": (IO.INT, {
|
||||
"default": 0, "min": 0, "max": 2147483647,
|
||||
"tooltip": "Random seed (0 for random)",
|
||||
}),
|
||||
"watermark": (IO.BOOLEAN, {
|
||||
"default": False,
|
||||
"tooltip": "Add watermark to output video",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
prompt,
|
||||
resolution="720P",
|
||||
ratio="16:9",
|
||||
duration=5,
|
||||
seed=0,
|
||||
watermark=False,
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("prompt is required for HappyHorse T2V")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
task_input = {"prompt": prompt.strip()}
|
||||
parameters = {
|
||||
"resolution": resolution,
|
||||
"ratio": ratio,
|
||||
"duration": duration,
|
||||
"seed": seed,
|
||||
}
|
||||
if watermark:
|
||||
parameters["watermark"] = True
|
||||
|
||||
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"HappyHorse T2V task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"HappyHorse_Text2Video": HappyHorseText2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"HappyHorse_Text2Video": "Modelverse HappyHorse T2V",
|
||||
}
|
||||
+167
@@ -0,0 +1,167 @@
|
||||
"""
|
||||
Kling V3 - Unified text/image-to-video and motion control model
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.kling_common import (
|
||||
ASPECT_RATIOS,
|
||||
CHARACTER_ORIENTATIONS,
|
||||
KLING_V3_TYPES,
|
||||
MODEL_KLING_V3,
|
||||
MODES,
|
||||
SOUNDS,
|
||||
YES_NO,
|
||||
)
|
||||
from .modelverse_api.requests.kling_v3 import KlingV3
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class KlingV3Node:
|
||||
"""
|
||||
Kling V3 unified video generation.
|
||||
Auto-routes to text-to-video, image-to-video, or motion control.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"negative_prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Negative prompt to avoid unwanted content",
|
||||
}),
|
||||
"kling_v3_type": (KLING_V3_TYPES, {
|
||||
"default": "auto",
|
||||
"tooltip": "Task type; auto infers from inputs (image→i2v, video→motion_control)",
|
||||
}),
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image (image-to-video / motion control)"}),
|
||||
"first_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "First frame URL (use either this OR first_frame_image, not both)",
|
||||
}),
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image (image-to-video)"}),
|
||||
"last_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "Last frame URL (use either this OR last_frame_image, not both)",
|
||||
}),
|
||||
"reference_video_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "Reference video URL (motion control; auto-selects motion_control when set)",
|
||||
}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "Output aspect ratio"}),
|
||||
"duration": (IO.INT, {
|
||||
"default": 5, "min": 3, "max": 15, "step": 1,
|
||||
"tooltip": "Video duration in seconds (3–15)",
|
||||
}),
|
||||
"mode": (MODES, {"default": "std", "tooltip": "std: 720p, pro: 1080p"}),
|
||||
"sound": (SOUNDS, {"default": "off", "tooltip": "Generate synchronized audio"}),
|
||||
"shot_type": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "Optional shot type, e.g. 'multi' for multi-shot",
|
||||
}),
|
||||
"character_orientation": (CHARACTER_ORIENTATIONS, {
|
||||
"default": "image",
|
||||
"tooltip": "Motion control: follow image or video orientation",
|
||||
}),
|
||||
"keep_original_sound": (YES_NO, {
|
||||
"default": "no",
|
||||
"tooltip": "Motion control: keep original reference video sound",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Kling"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
prompt,
|
||||
negative_prompt="",
|
||||
kling_v3_type="auto",
|
||||
first_frame_image=None,
|
||||
first_frame_url="",
|
||||
last_frame_image=None,
|
||||
last_frame_url="",
|
||||
reference_video_url="",
|
||||
aspect_ratio="16:9",
|
||||
duration=5,
|
||||
mode="std",
|
||||
sound="off",
|
||||
shot_type="",
|
||||
character_orientation="image",
|
||||
keep_original_sound="no",
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
request = KlingV3(
|
||||
prompt=prompt,
|
||||
negative_prompt=negative_prompt,
|
||||
kling_v3_type=kling_v3_type,
|
||||
first_frame=first_frame_image,
|
||||
first_frame_url=first_frame_url,
|
||||
last_frame=last_frame_image,
|
||||
last_frame_url=last_frame_url,
|
||||
reference_video_url=reference_video_url,
|
||||
aspect_ratio=aspect_ratio,
|
||||
duration=duration,
|
||||
mode=mode,
|
||||
sound=sound,
|
||||
shot_type=shot_type,
|
||||
character_orientation=character_orientation,
|
||||
keep_original_sound=keep_original_sound,
|
||||
)
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
print(f"Submitting Kling V3 task: model={MODEL_KLING_V3}, type={kling_v3_type}, prompt={prompt!r}")
|
||||
submit_res = mv_client.submit_task_request(request)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Kling V3 task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Kling_V3": KlingV3Node,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Kling_V3": "Modelverse Kling V3",
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
"""
|
||||
Kling V3 Omni - Multimodal video generation and editing model
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.kling_common import (
|
||||
ASPECT_RATIOS,
|
||||
MODEL_KLING_V3_OMNI,
|
||||
MODES,
|
||||
REFER_TYPES,
|
||||
SOUNDS,
|
||||
YES_NO,
|
||||
)
|
||||
from .modelverse_api.requests.kling_v3_omni import KlingV3Omni
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class KlingV3OmniNode:
|
||||
"""
|
||||
Kling V3 Omni multimodal video generation.
|
||||
Supports reference images, videos, and element library IDs.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"negative_prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Negative prompt to avoid unwanted content",
|
||||
}),
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame reference image"}),
|
||||
"first_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "First frame URL (use either this OR first_frame_image, not both)",
|
||||
}),
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "End frame reference image"}),
|
||||
"last_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "End frame URL (use either this OR last_frame_image, not both)",
|
||||
}),
|
||||
"reference_video_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "Reference video URL for editing or style reference",
|
||||
}),
|
||||
"refer_type": (REFER_TYPES, {
|
||||
"default": "feature",
|
||||
"tooltip": "feature: style/motion reference; base: video editing",
|
||||
}),
|
||||
"keep_original_sound": (YES_NO, {
|
||||
"default": "no",
|
||||
"tooltip": "Keep original audio from reference video",
|
||||
}),
|
||||
"element_id": (IO.INT, {
|
||||
"default": 0, "min": 0,
|
||||
"tooltip": "Kling element library ID (0 to skip)",
|
||||
}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {
|
||||
"default": "16:9",
|
||||
"tooltip": "Output aspect ratio (required without first-frame or video editing)",
|
||||
}),
|
||||
"duration": (IO.INT, {
|
||||
"default": 5, "min": 3, "max": 15, "step": 1,
|
||||
"tooltip": "Video duration in seconds (3–15)",
|
||||
}),
|
||||
"mode": (MODES, {"default": "std", "tooltip": "std: 720p, pro: 1080p"}),
|
||||
"sound": (SOUNDS, {
|
||||
"default": "off",
|
||||
"tooltip": "Generate synchronized audio (must be off when reference_video_url is set)",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Kling"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
prompt,
|
||||
negative_prompt="",
|
||||
first_frame_image=None,
|
||||
first_frame_url="",
|
||||
last_frame_image=None,
|
||||
last_frame_url="",
|
||||
reference_video_url="",
|
||||
refer_type="feature",
|
||||
keep_original_sound="no",
|
||||
element_id=0,
|
||||
aspect_ratio="16:9",
|
||||
duration=5,
|
||||
mode="std",
|
||||
sound="off",
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
request = KlingV3Omni(
|
||||
prompt=prompt,
|
||||
negative_prompt=negative_prompt,
|
||||
first_frame=first_frame_image,
|
||||
first_frame_url=first_frame_url,
|
||||
last_frame=last_frame_image,
|
||||
last_frame_url=last_frame_url,
|
||||
reference_video_url=reference_video_url,
|
||||
refer_type=refer_type,
|
||||
keep_original_sound=keep_original_sound,
|
||||
element_id=element_id,
|
||||
aspect_ratio=aspect_ratio,
|
||||
duration=duration,
|
||||
mode=mode,
|
||||
sound=sound,
|
||||
)
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
print(f"Submitting Kling V3 Omni task: model={MODEL_KLING_V3_OMNI}, prompt={prompt!r}")
|
||||
submit_res = mv_client.submit_task_request(request)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Kling V3 Omni task submitted: {task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Kling_V3_Omni": KlingV3OmniNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Kling_V3_Omni": "Modelverse Kling V3 Omni",
|
||||
}
|
||||
+56
-46
@@ -4,47 +4,43 @@ import asyncio
|
||||
from .utils import BaseRequest
|
||||
|
||||
|
||||
import asyncio
|
||||
from .utils import BaseRequest
|
||||
|
||||
class ModelverseClient:
|
||||
"""
|
||||
UCloud Modelverse API Client
|
||||
|
||||
This class handles the core communication with the Modelverse API.
|
||||
"""
|
||||
|
||||
BASE_URL = "https://api.modelverse.cn"
|
||||
API_PATH = "/v1/images/generations"
|
||||
|
||||
def __init__(self, api_key):
|
||||
"""
|
||||
Initialize Modelverse API client
|
||||
|
||||
Args:
|
||||
api_key (str): Modelverse API key
|
||||
"""
|
||||
self.api_key = api_key
|
||||
self.headers = {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json"
|
||||
}
|
||||
|
||||
self.headers = {"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json"}
|
||||
def post(self, endpoint, payload, timeout=180):
|
||||
url = f"{self.BASE_URL}{endpoint}"
|
||||
response = requests.post(url, headers=self.headers, json=payload, timeout=timeout)
|
||||
return self._handle_response(response)
|
||||
|
||||
def post(self, payload, timeout=180):
|
||||
"""
|
||||
Send POST request to Modelverse API
|
||||
def post_multipart(self, endpoint, data=None, files=None, timeout=180):
|
||||
"""POST with multipart/form-data. Content-Type is set by requests automatically."""
|
||||
url = f"{self.BASE_URL}{endpoint}"
|
||||
# Do not set Content-Type explicitly when using files; requests will handle it.
|
||||
headers = {k: v for k, v in self.headers.items() if k.lower() != "content-type"}
|
||||
response = requests.post(url, headers=headers, data=data, files=files, timeout=timeout)
|
||||
return self._handle_response(response)
|
||||
|
||||
Args:
|
||||
endpoint (str): API endpoint
|
||||
payload (dict): Request payload
|
||||
timeout (float, optional): Request timeout in seconds
|
||||
|
||||
Returns:
|
||||
dict: API response
|
||||
"""
|
||||
url = f"{self.BASE_URL}{self.API_PATH}"
|
||||
response = requests.post(
|
||||
url, headers=self.headers, json=payload, timeout=timeout)
|
||||
def get(self, endpoint, params=None, timeout=180):
|
||||
url = f"{self.BASE_URL}{endpoint}"
|
||||
headers = {"Authorization": f"Bearer {self.api_key}"}
|
||||
response = requests.get(url, headers=headers, params=params, timeout=timeout)
|
||||
return self._handle_response(response)
|
||||
|
||||
def _handle_response(self, response):
|
||||
if response.status_code == 401:
|
||||
raise Exception("Unauthorized: Invalid API key")
|
||||
|
||||
|
||||
# For backward compatibility with older error formats
|
||||
if response.status_code != 200:
|
||||
error_message = f"Error: {response.status_code}"
|
||||
try:
|
||||
@@ -60,26 +56,40 @@ class ModelverseClient:
|
||||
if response_data['code'] == 401:
|
||||
raise Exception("Unauthorized: Invalid API key")
|
||||
if response_data['code'] != 200:
|
||||
raise Exception(
|
||||
f"API Error: {response_data.get('message', 'Unknown error')}")
|
||||
raise Exception(f"API Error: {response_data.get('message', 'Unknown error')}")
|
||||
return response_data.get('data', {})
|
||||
return response_data
|
||||
|
||||
# --- New Methods for T2V ---
|
||||
def submit_task(self, model, task_input, parameters):
|
||||
endpoint = "/v1/tasks/submit"
|
||||
payload = {
|
||||
"model": model,
|
||||
"input": task_input,
|
||||
"parameters": parameters
|
||||
}
|
||||
return self.post(endpoint, payload)
|
||||
|
||||
def submit_task_request(self, request: BaseRequest):
|
||||
return self.post(request.API_PATH, request.build_payload())
|
||||
|
||||
def get_task_status(self, task_id):
|
||||
endpoint = f"/v1/tasks/status"
|
||||
params = {"task_id": task_id}
|
||||
return self.get(endpoint, params=params)
|
||||
|
||||
# --- Restored Async Methods for existing nodes ---
|
||||
async def async_send_request(self, request: BaseRequest):
|
||||
"""
|
||||
Sends an API request using a request object.
|
||||
|
||||
Args:
|
||||
request (BaseRequest): The request object containing payload and endpoint logic.
|
||||
|
||||
Returns:
|
||||
dict: API response or task result.
|
||||
"""
|
||||
payload = request.build_payload()
|
||||
if "seed" in payload:
|
||||
payload["seed"] = payload["seed"] % 2147483647 if payload["seed"] != -1 else -1
|
||||
|
||||
response = self.post(payload)
|
||||
endpoint = request.API_PATH
|
||||
# Support multipart form requests when available
|
||||
if hasattr(request, "build_multipart") and callable(getattr(request, "build_multipart")):
|
||||
data, files = request.build_multipart()
|
||||
response = self.post_multipart(endpoint, data=data, files=files)
|
||||
else:
|
||||
payload = request.build_payload()
|
||||
if isinstance(payload, dict) and "seed" in payload:
|
||||
payload["seed"] = payload["seed"] % 2147483647 if payload["seed"] != -1 else -1
|
||||
response = self.post(endpoint, payload)
|
||||
return response.get("data", [])
|
||||
|
||||
async def run_tasks(self, tasks):
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
"""
|
||||
Doubao Seedance 2.0 task submit request builder.
|
||||
API: https://www.compshare.cn/docs/modelverse/models/video_api/doubao-seedance-2-0
|
||||
"""
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from pydantic import Field
|
||||
from torch import Tensor
|
||||
|
||||
from ..utils import BaseRequest, image_to_base64
|
||||
|
||||
|
||||
MODEL = "doubao-seedance-2-0-260128"
|
||||
RESOLUTIONS = ["480p", "720p", "1080p"]
|
||||
RATIOS = ["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"]
|
||||
|
||||
|
||||
def _resolve_image(image: Optional[Tensor], url: str, label: str) -> Optional[str]:
|
||||
has_url = url and url.strip()
|
||||
has_image = image is not None
|
||||
if has_url and has_image:
|
||||
raise ValueError(f"{label}: provide either image or url, not both")
|
||||
if has_url:
|
||||
return url.strip()
|
||||
if has_image:
|
||||
encoded = image_to_base64(image)
|
||||
if not encoded:
|
||||
raise ValueError(f"{label}: failed to convert image to base64")
|
||||
return encoded
|
||||
return None
|
||||
|
||||
|
||||
def _normalize_prompt(value) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, list):
|
||||
value = value[0] if value else ""
|
||||
if isinstance(value, dict):
|
||||
return ""
|
||||
return str(value).strip()
|
||||
|
||||
|
||||
def _image_content(url: str, role: Optional[str] = None) -> Dict[str, Any]:
|
||||
item: Dict[str, Any] = {
|
||||
"type": "image_url",
|
||||
"image_url": {"url": url},
|
||||
}
|
||||
if role:
|
||||
item["role"] = role
|
||||
return item
|
||||
|
||||
|
||||
class DoubaoSeedance2(BaseRequest):
|
||||
"""
|
||||
Doubao Seedance 2.0 video generation task request.
|
||||
Supports text, first/last frame images, reference image/video/audio.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/tasks/submit"
|
||||
|
||||
prompt: str = Field(default="", description="Text prompt for video generation")
|
||||
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
|
||||
first_frame_url: str = Field(default="", description="First frame image URL or base64")
|
||||
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
|
||||
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
|
||||
reference_image: Optional[Tensor] = Field(default=None, description="Reference image tensor")
|
||||
reference_image_url: str = Field(default="", description="Reference image URL or base64")
|
||||
reference_video_url: str = Field(default="", description="Reference video URL")
|
||||
reference_audio_url: str = Field(default="", description="Reference audio URL")
|
||||
duration: int = Field(default=5, ge=4, le=15, description="Video duration in seconds")
|
||||
resolution: str = Field(default="720p", description="Output resolution")
|
||||
ratio: str = Field(default="adaptive", description="Aspect ratio")
|
||||
seed: int = Field(default=0, ge=0, le=2147483647, description="Random seed")
|
||||
generate_audio: bool = Field(default=False, description="Whether to generate synchronized audio")
|
||||
camera_fixed: bool = Field(default=False, description="Whether to fix camera position")
|
||||
watermark: bool = Field(default=False, description="Whether to add watermark")
|
||||
draft: bool = Field(default=False, description="Draft mode (480p only)")
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str = "",
|
||||
first_frame: Optional[Tensor] = None,
|
||||
first_frame_url: str = "",
|
||||
last_frame: Optional[Tensor] = None,
|
||||
last_frame_url: str = "",
|
||||
reference_image: Optional[Tensor] = None,
|
||||
reference_image_url: str = "",
|
||||
reference_video_url: str = "",
|
||||
reference_audio_url: str = "",
|
||||
duration: int = 5,
|
||||
resolution: str = "720p",
|
||||
ratio: str = "adaptive",
|
||||
seed: int = 0,
|
||||
generate_audio: bool = False,
|
||||
camera_fixed: bool = False,
|
||||
watermark: bool = False,
|
||||
draft: bool = False,
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = _normalize_prompt(prompt)
|
||||
self.first_frame = first_frame
|
||||
self.first_frame_url = first_frame_url
|
||||
self.last_frame = last_frame
|
||||
self.last_frame_url = last_frame_url
|
||||
self.reference_image = reference_image
|
||||
self.reference_image_url = reference_image_url
|
||||
self.reference_video_url = reference_video_url
|
||||
self.reference_audio_url = reference_audio_url
|
||||
self.duration = duration
|
||||
self.resolution = resolution
|
||||
self.ratio = ratio
|
||||
self.seed = seed
|
||||
self.generate_audio = generate_audio
|
||||
self.camera_fixed = camera_fixed
|
||||
self.watermark = watermark
|
||||
self.draft = draft
|
||||
|
||||
def build_content(self) -> List[Dict[str, Any]]:
|
||||
if self.draft and self.resolution != "480p":
|
||||
raise ValueError("Draft mode only supports 480p resolution")
|
||||
|
||||
first_url = _resolve_image(self.first_frame, self.first_frame_url, "First frame")
|
||||
last_url = _resolve_image(self.last_frame, self.last_frame_url, "Last frame")
|
||||
ref_img_url = _resolve_image(self.reference_image, self.reference_image_url, "Reference image")
|
||||
|
||||
content: List[Dict[str, Any]] = []
|
||||
if self.prompt and self.prompt.strip():
|
||||
content.append({"type": "text", "text": self.prompt.strip()})
|
||||
|
||||
frame_images = [(first_url, "first_frame"), (last_url, "last_frame")]
|
||||
frame_images = [(url, role) for url, role in frame_images if url]
|
||||
if ref_img_url:
|
||||
frame_images.append((ref_img_url, "reference_image"))
|
||||
|
||||
for url, role in frame_images:
|
||||
content.append(_image_content(url, role))
|
||||
|
||||
if self.reference_video_url and self.reference_video_url.strip():
|
||||
content.append({
|
||||
"type": "video_url",
|
||||
"video_url": {"url": self.reference_video_url.strip()},
|
||||
"role": "reference_video",
|
||||
})
|
||||
|
||||
if self.reference_audio_url and self.reference_audio_url.strip():
|
||||
content.append({
|
||||
"type": "audio_url",
|
||||
"audio_url": {"url": self.reference_audio_url.strip()},
|
||||
"role": "reference_audio",
|
||||
})
|
||||
|
||||
if not content:
|
||||
raise ValueError(
|
||||
"At least a text prompt or one input asset (image/video/audio) is required. "
|
||||
f"Current prompt is empty (received: {self.prompt!r})"
|
||||
)
|
||||
return content
|
||||
|
||||
def build_parameters(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"duration": self.duration,
|
||||
"resolution": self.resolution,
|
||||
"ratio": self.ratio,
|
||||
"seed": self.seed,
|
||||
"generate_audio": self.generate_audio,
|
||||
"camera_fixed": self.camera_fixed,
|
||||
"watermark": self.watermark,
|
||||
"draft": self.draft,
|
||||
}
|
||||
|
||||
def build_payload(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"model": MODEL,
|
||||
"input": {"content": self.build_content()},
|
||||
"parameters": self.build_parameters(),
|
||||
}
|
||||
|
||||
def field_required(self):
|
||||
return []
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"prompt",
|
||||
"first_frame",
|
||||
"last_frame",
|
||||
"reference_image",
|
||||
"reference_video_url",
|
||||
"reference_audio_url",
|
||||
"duration",
|
||||
"resolution",
|
||||
"ratio",
|
||||
"seed",
|
||||
"generate_audio",
|
||||
"camera_fixed",
|
||||
"watermark",
|
||||
"draft",
|
||||
]
|
||||
@@ -8,6 +8,7 @@ class FluxDev(BaseRequest):
|
||||
"""
|
||||
Flux-dev text to image model, 12 billion parameter rectified flow transformer
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
guidance_scale: Optional[float] = Field(
|
||||
default=2.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
|
||||
num_images: Optional[int] = Field(
|
||||
|
||||
@@ -8,6 +8,7 @@ class FluxKontextMax(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Max for image editing.
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
guidance_scale: Optional[float] = Field(
|
||||
default=2.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
|
||||
image: Tensor = Field(...,
|
||||
@@ -60,6 +61,7 @@ class FluxKontextMaxMulti(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Max for multiple image inputs.
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
guidance_scale: Optional[float] = Field(
|
||||
default=3.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
|
||||
images: list[Tensor] = Field(...,
|
||||
@@ -112,6 +114,7 @@ class FluxKontextMaxT2I(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Pro text-to-image model
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
prompt: str = Field(...,
|
||||
description="The prompt to generate an image from.")
|
||||
aspect_ratio: Optional[str] = Field(
|
||||
|
||||
@@ -8,6 +8,7 @@ class FluxKontextPro(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Pro for image editing.
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
prompt: str = Field(...,
|
||||
description="The prompt to generate an image from.")
|
||||
image: Tensor = Field(...,
|
||||
@@ -61,6 +62,7 @@ class FluxKontextProMulti(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Pro for multiple image inputs.
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
prompt: str = Field(...,
|
||||
description="The prompt to generate an image from.")
|
||||
images: list[Tensor] = Field(...,
|
||||
@@ -113,6 +115,7 @@ class FluxKontextProT2I(BaseRequest):
|
||||
"""
|
||||
Flux Kontext Pro text-to-image model
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
prompt: str = Field(...,
|
||||
description="The prompt to generate an image from.")
|
||||
aspect_ratio: Optional[str] = Field(
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
from typing import Optional, List, Dict, Any
|
||||
from pydantic import Field
|
||||
from ..utils import BaseRequest
|
||||
from torch import Tensor
|
||||
import base64
|
||||
import io
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
|
||||
def _tensor_to_base64(image: Tensor, mime_type: str = "image/png") -> Dict[str, str]:
|
||||
"""Convert a ComfyUI image tensor (HWC or 4D with batch) to base64 without data URI."""
|
||||
if image is None:
|
||||
return None
|
||||
|
||||
# If batch, take first frame
|
||||
if hasattr(image, 'shape') and len(image.shape) == 4:
|
||||
image = image[0]
|
||||
|
||||
# Tensor (H, W, C) in 0..1 -> uint8
|
||||
np_img = np.clip(255.0 * image.cpu().numpy(), 0, 255).astype(np.uint8)
|
||||
pil_img = Image.fromarray(np_img)
|
||||
|
||||
fmt = 'PNG' if mime_type.lower().endswith('png') else 'JPEG'
|
||||
with io.BytesIO() as bio:
|
||||
pil_img.save(bio, format=fmt)
|
||||
data = bio.getvalue()
|
||||
|
||||
return {
|
||||
"mimeType": mime_type,
|
||||
"data": base64.b64encode(data).decode("utf-8"),
|
||||
}
|
||||
|
||||
|
||||
class GeminiFlashImageRequest(BaseRequest):
|
||||
"""
|
||||
Request builder for Gemini Flash Image generateContent endpoint.
|
||||
Supports text-to-image and image-edit (text+image to image).
|
||||
"""
|
||||
|
||||
API_PATH_TEMPLATE = "/v1beta/models/{model}:generateContent"
|
||||
|
||||
prompt: str = Field(..., description="Text prompt")
|
||||
model: str = Field(default="gemini-3.1-flash-image", description="Gemini Flash Image model")
|
||||
image: Optional[Tensor] = Field(default=None, description="Optional input image for edit")
|
||||
mime_type: str = Field(default="image/png", description="MIME type for inline image data")
|
||||
|
||||
def __init__(self, prompt: str, model: str = "gemini-3.1-flash-image", image: Optional[Tensor] = None, mime_type: str = "image/png", **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.model = model
|
||||
self.API_PATH = self.API_PATH_TEMPLATE.format(model=model)
|
||||
self.image = image
|
||||
self.mime_type = mime_type
|
||||
|
||||
def build_payload(self) -> Dict[str, Any]:
|
||||
parts: List[Dict[str, Any]] = []
|
||||
if self.prompt:
|
||||
parts.append({"text": self.prompt})
|
||||
|
||||
if isinstance(self.image, Tensor):
|
||||
parts.append({
|
||||
"inlineData": _tensor_to_base64(self.image, self.mime_type)
|
||||
})
|
||||
|
||||
payload = {
|
||||
"contents": [
|
||||
{
|
||||
"role": "user",
|
||||
"parts": parts,
|
||||
}
|
||||
],
|
||||
"generationConfig": {
|
||||
"responseModalities": ["TEXT", "IMAGE"],
|
||||
},
|
||||
}
|
||||
return self._remove_empty_fields(payload)
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt"]
|
||||
|
||||
def field_order(self):
|
||||
return ["model", "prompt", "image", "mime_type"]
|
||||
@@ -0,0 +1,121 @@
|
||||
"""
|
||||
Gemini 3 Pro Image (Nano Banana Pro) request builder.
|
||||
Supports text-to-image and image editing with advanced features like
|
||||
aspect ratio, resolution control, and Google Search grounding.
|
||||
"""
|
||||
import io
|
||||
import base64
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from typing import Optional, List, Dict, Any
|
||||
from pydantic import Field
|
||||
from torch import Tensor
|
||||
|
||||
from ..utils import BaseRequest
|
||||
|
||||
|
||||
def _tensor_to_base64(image: Tensor, mime_type: str = "image/png") -> Dict[str, str]:
|
||||
"""Convert a ComfyUI image tensor (HWC or 4D with batch) to base64 without data URI."""
|
||||
if image is None:
|
||||
return None
|
||||
|
||||
# If batch, take first frame
|
||||
if hasattr(image, 'shape') and len(image.shape) == 4:
|
||||
image = image[0]
|
||||
|
||||
# Tensor (H, W, C) in 0..1 -> uint8
|
||||
np_img = np.clip(255.0 * image.cpu().numpy(), 0, 255).astype(np.uint8)
|
||||
pil_img = Image.fromarray(np_img)
|
||||
|
||||
fmt = 'PNG' if mime_type.lower().endswith('png') else 'JPEG'
|
||||
with io.BytesIO() as bio:
|
||||
pil_img.save(bio, format=fmt)
|
||||
data = bio.getvalue()
|
||||
|
||||
return {
|
||||
"mimeType": mime_type,
|
||||
"data": base64.b64encode(data).decode("utf-8"),
|
||||
}
|
||||
|
||||
|
||||
class GeminiProImageRequest(BaseRequest):
|
||||
"""
|
||||
Request builder for Gemini 3 Pro Image (Nano Banana Pro) generateContent endpoint.
|
||||
Supports text-to-image, image editing, and advanced features like aspect ratio,
|
||||
resolution control (1K/2K/4K), and Google Search grounding.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1beta/models/gemini-3-pro-image:generateContent"
|
||||
|
||||
prompt: str = Field(..., description="Text prompt")
|
||||
images: Optional[List[Tensor]] = Field(default=None, description="Optional input images (up to 14)")
|
||||
mime_type: str = Field(default="image/png", description="MIME type for inline image data")
|
||||
aspect_ratio: Optional[str] = Field(default=None, description="Aspect ratio: 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9")
|
||||
image_size: Optional[str] = Field(default=None, description="Image resolution: 1K, 2K, 4K")
|
||||
use_google_search: bool = Field(default=False, description="Enable Google Search grounding")
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str,
|
||||
images: Optional[List[Tensor]] = None,
|
||||
mime_type: str = "image/png",
|
||||
aspect_ratio: Optional[str] = None,
|
||||
image_size: Optional[str] = None,
|
||||
use_google_search: bool = False,
|
||||
**kwargs
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.images = images
|
||||
self.mime_type = mime_type
|
||||
self.aspect_ratio = aspect_ratio
|
||||
self.image_size = image_size
|
||||
self.use_google_search = use_google_search
|
||||
|
||||
def build_payload(self) -> Dict[str, Any]:
|
||||
parts: List[Dict[str, Any]] = []
|
||||
|
||||
# Add text prompt
|
||||
if self.prompt:
|
||||
parts.append({"text": self.prompt})
|
||||
|
||||
# Add images (up to 14)
|
||||
if self.images:
|
||||
for img in self.images[:14]: # Limit to 14 images
|
||||
if isinstance(img, Tensor):
|
||||
parts.append({
|
||||
"inlineData": _tensor_to_base64(img, self.mime_type)
|
||||
})
|
||||
|
||||
payload = {
|
||||
"contents": [
|
||||
{
|
||||
"role": "user",
|
||||
"parts": parts,
|
||||
}
|
||||
],
|
||||
"generationConfig": {
|
||||
"responseModalities": ["TEXT", "IMAGE"],
|
||||
},
|
||||
}
|
||||
|
||||
# Add imageConfig if aspect_ratio or image_size specified
|
||||
if self.aspect_ratio or self.image_size:
|
||||
image_config = {}
|
||||
if self.aspect_ratio:
|
||||
image_config["aspectRatio"] = self.aspect_ratio
|
||||
if self.image_size:
|
||||
image_config["imageSize"] = self.image_size
|
||||
payload["generationConfig"]["imageConfig"] = image_config
|
||||
|
||||
# Add Google Search tool if enabled
|
||||
if self.use_google_search:
|
||||
payload["tools"] = [{"google_search": {}}]
|
||||
|
||||
return self._remove_empty_fields(payload)
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt"]
|
||||
|
||||
def field_order(self):
|
||||
return ["prompt", "images", "mime_type", "aspect_ratio", "image_size", "use_google_search"]
|
||||
@@ -0,0 +1,65 @@
|
||||
from typing import Optional
|
||||
from pydantic import Field
|
||||
from ..utils import BaseRequest
|
||||
|
||||
|
||||
class GPTImage1(BaseRequest):
|
||||
"""
|
||||
gpt-image-1 text-to-image via /v1/images/generations.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/images/generations"
|
||||
|
||||
prompt: str = Field(..., description="Prompt text")
|
||||
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
|
||||
size: Optional[str] = Field(default="1024x1024", description="Output size: 1024x1024, 1024x1536, 1536x1024")
|
||||
quality: Optional[str] = Field(default=None, description="Image quality: low, medium, high")
|
||||
output_format: Optional[str] = Field(default="png", description="Output format: png or jpeg")
|
||||
output_compression: Optional[int] = Field(default=100, ge=0, le=100, description="Compression strength 0-100")
|
||||
# negative_prompt and response_format removed per API behavior
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str,
|
||||
num_images: Optional[int] = 1,
|
||||
size: Optional[str] = "1024x1024",
|
||||
quality: Optional[str] = None,
|
||||
output_format: Optional[str] = "png",
|
||||
output_compression: Optional[int] = 100,
|
||||
# guidance_scale removed per API behavior
|
||||
# negative_prompt/response_format removed per API behavior
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.num_images = num_images
|
||||
self.size = size
|
||||
self.quality = quality
|
||||
self.output_format = output_format
|
||||
self.output_compression = output_compression
|
||||
|
||||
def build_payload(self) -> dict:
|
||||
payload = {
|
||||
"model": "gpt-image-1",
|
||||
"prompt": self.prompt,
|
||||
"n": self.num_images,
|
||||
"size": self.size,
|
||||
"quality": self.quality,
|
||||
"output_format": self.output_format,
|
||||
"output_compression": self.output_compression,
|
||||
}
|
||||
return self._remove_empty_fields(payload)
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt"]
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"model",
|
||||
"prompt",
|
||||
"n",
|
||||
"size",
|
||||
"quality",
|
||||
"output_format",
|
||||
"output_compression",
|
||||
]
|
||||
@@ -0,0 +1,115 @@
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
from pydantic import Field
|
||||
from ..utils import BaseRequest, tensor2images
|
||||
from torch import Tensor
|
||||
import io
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def _tensor_to_png_file(image: Tensor, filename: str) -> Tuple[str, bytes, str] | None:
|
||||
"""Convert a ComfyUI tensor image (3D or 4D) to a PNG file tuple for requests files."""
|
||||
if image is None:
|
||||
return None
|
||||
# Keep batch if present; tensor2images expects a batch dimension
|
||||
try:
|
||||
if hasattr(image, 'shape') and len(image.shape) == 3:
|
||||
# add batch dim
|
||||
img_batch = image.unsqueeze(0)
|
||||
else:
|
||||
img_batch = image
|
||||
pil_img = tensor2images(img_batch)[0]
|
||||
except Exception:
|
||||
# Fallback: attempt naive conversion
|
||||
from PIL import Image as _Image
|
||||
import numpy as _np
|
||||
arr = (image.cpu().numpy() * 255.0).clip(0, 255).astype('uint8')
|
||||
pil_img = _Image.fromarray(arr)
|
||||
with io.BytesIO() as bio:
|
||||
pil_img.save(bio, format="PNG")
|
||||
data = bio.getvalue()
|
||||
return (filename, data, "image/png")
|
||||
|
||||
|
||||
class GPTImage1Edit(BaseRequest):
|
||||
"""
|
||||
gpt-image-1 image edit via /v1/images/edits using multipart/form-data.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/images/edits"
|
||||
|
||||
prompt: str = Field(..., description="Edit prompt")
|
||||
image: Tensor = Field(..., description="Input image")
|
||||
mask: Optional[Tensor] = Field(default=None, description="Optional mask image")
|
||||
|
||||
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
|
||||
size: Optional[str] = Field(default="1024x1024", description="Output size: 1024x1024, 1024x1536, 1536x1024")
|
||||
quality: Optional[str] = Field(default=None, description="Image quality: low, medium, high")
|
||||
output_format: Optional[str] = Field(default="png", description="Output format: png or jpeg")
|
||||
output_compression: Optional[int] = Field(default=100, ge=0, le=100, description="Compression strength 0-100")
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str,
|
||||
image: Tensor,
|
||||
mask: Optional[Tensor] = None,
|
||||
num_images: Optional[int] = 1,
|
||||
size: Optional[str] = "1024x1024",
|
||||
quality: Optional[str] = None,
|
||||
output_format: Optional[str] = "png",
|
||||
output_compression: Optional[int] = 100,
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.image = image
|
||||
self.mask = mask
|
||||
self.num_images = num_images
|
||||
self.size = size
|
||||
self.quality = quality
|
||||
self.output_format = output_format
|
||||
self.output_compression = output_compression
|
||||
|
||||
def build_multipart(self) -> Tuple[Dict[str, Any], Dict[str, Any]]:
|
||||
data = {
|
||||
"model": "gpt-image-1",
|
||||
"prompt": self.prompt,
|
||||
"n": self.num_images,
|
||||
"size": self.size,
|
||||
"output_format": self.output_format,
|
||||
"output_compression": str(self.output_compression) if self.output_compression is not None else None,
|
||||
}
|
||||
if self.quality:
|
||||
data["quality"] = self.quality
|
||||
# Filter None values
|
||||
data = {k: v for k, v in data.items() if v is not None and v != ""}
|
||||
|
||||
files = {}
|
||||
img_file = _tensor_to_png_file(self.image, "image.png")
|
||||
if img_file is not None:
|
||||
files["image"] = img_file
|
||||
if self.mask is not None:
|
||||
mask_file = _tensor_to_png_file(self.mask, "mask.png")
|
||||
if mask_file is not None:
|
||||
files["mask"] = mask_file
|
||||
|
||||
return data, files
|
||||
|
||||
# For interface completeness; not used by multipart path
|
||||
def build_payload(self) -> dict:
|
||||
return {}
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt", "image"]
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"model",
|
||||
"prompt",
|
||||
"image",
|
||||
"mask",
|
||||
"n",
|
||||
"size",
|
||||
"quality",
|
||||
"output_format",
|
||||
"output_compression",
|
||||
]
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Shared helpers for Kling V3 request builders."""
|
||||
import base64
|
||||
from typing import Optional
|
||||
|
||||
from torch import Tensor
|
||||
|
||||
from ..utils import encode_image, tensor2images
|
||||
|
||||
|
||||
MODEL_KLING_V3 = "kling-v3"
|
||||
MODEL_KLING_V3_OMNI = "kling-v3-omni"
|
||||
|
||||
MODES = ["std", "pro"]
|
||||
SOUNDS = ["off", "on"]
|
||||
ASPECT_RATIOS = ["16:9", "9:16", "1:1"]
|
||||
KLING_V3_TYPES = ["auto", "t2v", "i2v", "motion_control"]
|
||||
CHARACTER_ORIENTATIONS = ["image", "video"]
|
||||
YES_NO = ["yes", "no"]
|
||||
REFER_TYPES = ["feature", "base"]
|
||||
|
||||
|
||||
def normalize_kling_image_value(value: str) -> str:
|
||||
"""Kling accepts http(s) URLs or raw base64 without a data:image prefix."""
|
||||
value = value.strip()
|
||||
if value.startswith("data:image/") and "," in value:
|
||||
return value.split(",", 1)[1]
|
||||
return value
|
||||
|
||||
|
||||
def resolve_image(image: Optional[Tensor], url: str, label: str) -> Optional[str]:
|
||||
has_url = url and url.strip()
|
||||
has_image = image is not None
|
||||
if has_url and has_image:
|
||||
raise ValueError(f"{label}: provide either image or url, not both")
|
||||
if has_url:
|
||||
return normalize_kling_image_value(url)
|
||||
if has_image:
|
||||
data_bytes, _ = encode_image(tensor2images(image)[0])
|
||||
if not data_bytes:
|
||||
raise ValueError(f"{label}: failed to convert image to base64")
|
||||
return base64.b64encode(data_bytes).decode("utf-8")
|
||||
return None
|
||||
|
||||
|
||||
def normalize_prompt(value) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, list):
|
||||
value = value[0] if value else ""
|
||||
if isinstance(value, dict):
|
||||
return ""
|
||||
return str(value).strip()
|
||||
@@ -0,0 +1,172 @@
|
||||
"""
|
||||
Kling V3 task submit request builder.
|
||||
Routes to t2v / i2v / motion_control via input fields or parameters.kling_v3_type.
|
||||
"""
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from pydantic import Field
|
||||
from torch import Tensor
|
||||
|
||||
from ..utils import BaseRequest
|
||||
from .kling_common import (
|
||||
ASPECT_RATIOS,
|
||||
CHARACTER_ORIENTATIONS,
|
||||
KLING_V3_TYPES,
|
||||
MODEL_KLING_V3,
|
||||
MODES,
|
||||
SOUNDS,
|
||||
YES_NO,
|
||||
normalize_prompt,
|
||||
resolve_image,
|
||||
)
|
||||
|
||||
|
||||
class KlingV3(BaseRequest):
|
||||
"""
|
||||
Kling V3 unified video generation task request.
|
||||
Auto-routes to text-to-video, image-to-video, or motion control.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/tasks/submit"
|
||||
|
||||
prompt: str = Field(default="", description="Text prompt for video generation")
|
||||
negative_prompt: str = Field(default="", description="Negative prompt")
|
||||
kling_v3_type: str = Field(
|
||||
default="auto",
|
||||
description="Task variant: auto, t2v, i2v, motion_control",
|
||||
)
|
||||
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
|
||||
first_frame_url: str = Field(default="", description="First frame image URL or base64")
|
||||
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
|
||||
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
|
||||
reference_video_url: str = Field(default="", description="Reference video URL for motion control")
|
||||
aspect_ratio: str = Field(default="16:9", description="Aspect ratio")
|
||||
duration: int = Field(default=5, ge=3, le=15, description="Video duration in seconds")
|
||||
mode: str = Field(default="std", description="Generation mode: std or pro")
|
||||
sound: str = Field(default="off", description="Generate synchronized audio: on or off")
|
||||
shot_type: str = Field(default="", description="Shot type, e.g. multi for multi-shot")
|
||||
character_orientation: str = Field(
|
||||
default="image",
|
||||
description="Motion control orientation: image or video",
|
||||
)
|
||||
keep_original_sound: str = Field(
|
||||
default="no",
|
||||
description="Keep original sound from reference video: yes or no",
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str = "",
|
||||
negative_prompt: str = "",
|
||||
kling_v3_type: str = "auto",
|
||||
first_frame: Optional[Tensor] = None,
|
||||
first_frame_url: str = "",
|
||||
last_frame: Optional[Tensor] = None,
|
||||
last_frame_url: str = "",
|
||||
reference_video_url: str = "",
|
||||
aspect_ratio: str = "16:9",
|
||||
duration: int = 5,
|
||||
mode: str = "std",
|
||||
sound: str = "off",
|
||||
shot_type: str = "",
|
||||
character_orientation: str = "image",
|
||||
keep_original_sound: str = "no",
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = normalize_prompt(prompt)
|
||||
self.negative_prompt = normalize_prompt(negative_prompt)
|
||||
self.kling_v3_type = kling_v3_type
|
||||
self.first_frame = first_frame
|
||||
self.first_frame_url = first_frame_url
|
||||
self.last_frame = last_frame
|
||||
self.last_frame_url = last_frame_url
|
||||
self.reference_video_url = reference_video_url
|
||||
self.aspect_ratio = aspect_ratio
|
||||
self.duration = duration
|
||||
self.mode = mode
|
||||
self.sound = sound
|
||||
self.shot_type = shot_type
|
||||
self.character_orientation = character_orientation
|
||||
self.keep_original_sound = keep_original_sound
|
||||
|
||||
def _is_motion_control(self) -> bool:
|
||||
if self.kling_v3_type == "motion_control":
|
||||
return True
|
||||
if self.kling_v3_type == "auto" and self.reference_video_url and self.reference_video_url.strip():
|
||||
return True
|
||||
return False
|
||||
|
||||
def build_input(self) -> Dict[str, Any]:
|
||||
task_input: Dict[str, Any] = {}
|
||||
if self.prompt:
|
||||
task_input["prompt"] = self.prompt
|
||||
if self.negative_prompt:
|
||||
task_input["negative_prompt"] = self.negative_prompt
|
||||
|
||||
first_url = resolve_image(self.first_frame, self.first_frame_url, "First frame")
|
||||
last_url = resolve_image(self.last_frame, self.last_frame_url, "Last frame")
|
||||
|
||||
if self._is_motion_control():
|
||||
if not first_url:
|
||||
raise ValueError("Motion control requires a first frame image or URL")
|
||||
if not self.reference_video_url or not self.reference_video_url.strip():
|
||||
raise ValueError("Motion control requires reference_video_url")
|
||||
task_input["first_frame_url"] = first_url
|
||||
task_input["video_url"] = self.reference_video_url.strip()
|
||||
return task_input
|
||||
|
||||
if first_url:
|
||||
task_input["first_frame_url"] = first_url
|
||||
if last_url:
|
||||
task_input["last_frame_url"] = last_url
|
||||
|
||||
if not task_input.get("prompt") and not first_url:
|
||||
raise ValueError(
|
||||
"At least a prompt or first frame image is required. "
|
||||
f"Current prompt is empty (received: {self.prompt!r})"
|
||||
)
|
||||
return task_input
|
||||
|
||||
def build_parameters(self) -> Dict[str, Any]:
|
||||
params: Dict[str, Any] = {
|
||||
"duration": self.duration,
|
||||
"aspect_ratio": self.aspect_ratio,
|
||||
"mode": self.mode,
|
||||
"sound": self.sound,
|
||||
}
|
||||
if self.kling_v3_type and self.kling_v3_type != "auto":
|
||||
params["kling_v3_type"] = self.kling_v3_type
|
||||
if self.shot_type:
|
||||
params["shot_type"] = self.shot_type
|
||||
if self._is_motion_control():
|
||||
params["character_orientation"] = self.character_orientation
|
||||
params["keep_original_sound"] = self.keep_original_sound
|
||||
return params
|
||||
|
||||
def build_payload(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"model": MODEL_KLING_V3,
|
||||
"input": self.build_input(),
|
||||
"parameters": self.build_parameters(),
|
||||
}
|
||||
|
||||
def field_required(self):
|
||||
return []
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"prompt",
|
||||
"negative_prompt",
|
||||
"kling_v3_type",
|
||||
"first_frame",
|
||||
"last_frame",
|
||||
"reference_video_url",
|
||||
"aspect_ratio",
|
||||
"duration",
|
||||
"mode",
|
||||
"sound",
|
||||
"shot_type",
|
||||
"character_orientation",
|
||||
"keep_original_sound",
|
||||
]
|
||||
@@ -0,0 +1,155 @@
|
||||
"""
|
||||
Kling V3 Omni task submit request builder.
|
||||
Uses image_list / video_list / element_list parameters.
|
||||
"""
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from pydantic import Field
|
||||
from torch import Tensor
|
||||
|
||||
from ..utils import BaseRequest
|
||||
from .kling_common import (
|
||||
ASPECT_RATIOS,
|
||||
MODEL_KLING_V3_OMNI,
|
||||
MODES,
|
||||
REFER_TYPES,
|
||||
SOUNDS,
|
||||
YES_NO,
|
||||
normalize_prompt,
|
||||
resolve_image,
|
||||
)
|
||||
|
||||
|
||||
class KlingV3Omni(BaseRequest):
|
||||
"""
|
||||
Kling V3 Omni multimodal video generation task request.
|
||||
Supports reference images, videos, and element library IDs.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/tasks/submit"
|
||||
|
||||
prompt: str = Field(default="", description="Text prompt for video generation")
|
||||
negative_prompt: str = Field(default="", description="Negative prompt")
|
||||
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
|
||||
first_frame_url: str = Field(default="", description="First frame image URL or base64")
|
||||
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
|
||||
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
|
||||
reference_video_url: str = Field(default="", description="Reference video URL")
|
||||
refer_type: str = Field(default="feature", description="Video reference type: feature or base")
|
||||
keep_original_sound: str = Field(default="no", description="Keep original video sound: yes or no")
|
||||
element_id: int = Field(default=0, description="Element library ID (0 to skip)")
|
||||
aspect_ratio: str = Field(default="16:9", description="Aspect ratio")
|
||||
duration: int = Field(default=5, ge=3, le=15, description="Video duration in seconds")
|
||||
mode: str = Field(default="std", description="Generation mode: std or pro")
|
||||
sound: str = Field(default="off", description="Generate synchronized audio: on or off")
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str = "",
|
||||
negative_prompt: str = "",
|
||||
first_frame: Optional[Tensor] = None,
|
||||
first_frame_url: str = "",
|
||||
last_frame: Optional[Tensor] = None,
|
||||
last_frame_url: str = "",
|
||||
reference_video_url: str = "",
|
||||
refer_type: str = "feature",
|
||||
keep_original_sound: str = "no",
|
||||
element_id: int = 0,
|
||||
aspect_ratio: str = "16:9",
|
||||
duration: int = 5,
|
||||
mode: str = "std",
|
||||
sound: str = "off",
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = normalize_prompt(prompt)
|
||||
self.negative_prompt = normalize_prompt(negative_prompt)
|
||||
self.first_frame = first_frame
|
||||
self.first_frame_url = first_frame_url
|
||||
self.last_frame = last_frame
|
||||
self.last_frame_url = last_frame_url
|
||||
self.reference_video_url = reference_video_url
|
||||
self.refer_type = refer_type
|
||||
self.keep_original_sound = keep_original_sound
|
||||
self.element_id = element_id
|
||||
self.aspect_ratio = aspect_ratio
|
||||
self.duration = duration
|
||||
self.mode = mode
|
||||
self.sound = sound
|
||||
|
||||
def build_input(self) -> Dict[str, Any]:
|
||||
task_input: Dict[str, Any] = {}
|
||||
if self.prompt:
|
||||
task_input["prompt"] = self.prompt
|
||||
if self.negative_prompt:
|
||||
task_input["negative_prompt"] = self.negative_prompt
|
||||
if not task_input.get("prompt"):
|
||||
raise ValueError(
|
||||
"Prompt is required for Kling V3 Omni. "
|
||||
f"Current prompt is empty (received: {self.prompt!r})"
|
||||
)
|
||||
return task_input
|
||||
|
||||
def build_image_list(self) -> List[Dict[str, str]]:
|
||||
image_list: List[Dict[str, str]] = []
|
||||
first_url = resolve_image(self.first_frame, self.first_frame_url, "First frame")
|
||||
last_url = resolve_image(self.last_frame, self.last_frame_url, "Last frame")
|
||||
if first_url:
|
||||
image_list.append({"image_url": first_url, "type": "first_frame"})
|
||||
if last_url:
|
||||
image_list.append({"image_url": last_url, "type": "end_frame"})
|
||||
return image_list
|
||||
|
||||
def build_parameters(self) -> Dict[str, Any]:
|
||||
has_video = bool(self.reference_video_url and self.reference_video_url.strip())
|
||||
if has_video and self.sound == "on":
|
||||
raise ValueError("sound must be 'off' when reference_video_url is provided")
|
||||
|
||||
params: Dict[str, Any] = {
|
||||
"duration": self.duration,
|
||||
"aspect_ratio": self.aspect_ratio,
|
||||
"mode": self.mode,
|
||||
"sound": self.sound,
|
||||
}
|
||||
|
||||
image_list = self.build_image_list()
|
||||
if image_list:
|
||||
params["image_list"] = image_list
|
||||
|
||||
if has_video:
|
||||
params["video_list"] = [{
|
||||
"video_url": self.reference_video_url.strip(),
|
||||
"refer_type": self.refer_type,
|
||||
"keep_original_sound": self.keep_original_sound,
|
||||
}]
|
||||
|
||||
if self.element_id:
|
||||
params["element_list"] = [{"element_id": self.element_id}]
|
||||
|
||||
return params
|
||||
|
||||
def build_payload(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"model": MODEL_KLING_V3_OMNI,
|
||||
"input": self.build_input(),
|
||||
"parameters": self.build_parameters(),
|
||||
}
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt"]
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"prompt",
|
||||
"negative_prompt",
|
||||
"first_frame",
|
||||
"last_frame",
|
||||
"reference_video_url",
|
||||
"refer_type",
|
||||
"keep_original_sound",
|
||||
"element_id",
|
||||
"aspect_ratio",
|
||||
"duration",
|
||||
"mode",
|
||||
"sound",
|
||||
]
|
||||
@@ -0,0 +1,73 @@
|
||||
from typing import Optional
|
||||
from pydantic import Field
|
||||
from ..utils import BaseRequest
|
||||
|
||||
|
||||
class QwenImage(BaseRequest):
|
||||
"""
|
||||
Qwen/Qwen-Image text-to-image via /v1/images/generations.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/images/generations"
|
||||
|
||||
prompt: str = Field(..., description="Prompt text")
|
||||
aspect_ratio: Optional[str] = Field(default="1:1", description="Aspect ratio e.g. 16:9, 1:1")
|
||||
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
|
||||
seed: Optional[int] = Field(default=-1, description="Random seed (-1 for random)")
|
||||
steps: Optional[int] = Field(default=20, ge=1, le=50, description="Inference steps")
|
||||
guidance_scale: Optional[float] = Field(default=2.5, ge=1.0, le=10.0, description="Guidance scale 1~10")
|
||||
negative_prompt: Optional[str] = Field(default="", description="Negative prompt")
|
||||
response_format: Optional[str] = Field(default="url", description='"url" or "b64_json"')
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str,
|
||||
aspect_ratio: Optional[str] = "1:1",
|
||||
num_images: Optional[int] = 1,
|
||||
seed: Optional[int] = -1,
|
||||
steps: Optional[int] = 20,
|
||||
guidance_scale: Optional[float] = 2.5,
|
||||
negative_prompt: Optional[str] = "",
|
||||
response_format: Optional[str] = "url",
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.aspect_ratio = aspect_ratio
|
||||
self.num_images = num_images
|
||||
self.seed = seed
|
||||
self.steps = steps
|
||||
self.guidance_scale = guidance_scale
|
||||
self.negative_prompt = negative_prompt
|
||||
self.response_format = response_format
|
||||
|
||||
def build_payload(self) -> dict:
|
||||
payload = {
|
||||
"model": "Qwen/Qwen-Image",
|
||||
"prompt": self.prompt,
|
||||
"aspect_ratio": self.aspect_ratio,
|
||||
"n": self.num_images,
|
||||
"seed": self.seed,
|
||||
"steps": self.steps,
|
||||
"guidance_scale": self.guidance_scale,
|
||||
"negative_prompt": self.negative_prompt,
|
||||
"response_format": self.response_format,
|
||||
}
|
||||
return self._remove_empty_fields(payload)
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt"]
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"model",
|
||||
"prompt",
|
||||
"aspect_ratio",
|
||||
"n",
|
||||
"seed",
|
||||
"steps",
|
||||
"guidance_scale",
|
||||
"negative_prompt",
|
||||
"response_format",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
from typing import Optional
|
||||
from pydantic import Field
|
||||
from ..utils import BaseRequest, image_to_base64
|
||||
from torch import Tensor
|
||||
|
||||
|
||||
class QwenImageEdit(BaseRequest):
|
||||
"""
|
||||
Qwen/Qwen-Image-Edit request model for /v1/images/generations (OpenAI-compatible).
|
||||
Supports image edit with a prompt and optional controls.
|
||||
"""
|
||||
|
||||
API_PATH = "/v1/images/generations"
|
||||
|
||||
prompt: str = Field(..., description="Prompt text for the edit")
|
||||
image: Tensor = Field(..., description="Input image tensor for edit")
|
||||
negative_prompt: Optional[str] = Field(default="", description="Negative prompt")
|
||||
|
||||
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images to generate (n)")
|
||||
strength: Optional[float] = Field(default=0.8, ge=0.0, le=1.0, description="Edit strength 0~1")
|
||||
seed: Optional[int] = Field(default=-1, description="Random seed (-1 for random)")
|
||||
steps: Optional[int] = Field(default=20, ge=1, le=50, description="Inference steps 1~50")
|
||||
guidance_scale: Optional[float] = Field(default=2.5, ge=1.0, le=10.0, description="Guidance scale 1~10")
|
||||
response_format: Optional[str] = Field(default="url", description='"url" or "b64_json"')
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
prompt: str,
|
||||
image: Tensor,
|
||||
negative_prompt: Optional[str] = "",
|
||||
num_images: Optional[int] = 1,
|
||||
strength: Optional[float] = 0.8,
|
||||
seed: Optional[int] = -1,
|
||||
steps: Optional[int] = 20,
|
||||
guidance_scale: Optional[float] = 2.5,
|
||||
response_format: Optional[str] = "url",
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
self.prompt = prompt
|
||||
self.negative_prompt = negative_prompt
|
||||
self.num_images = num_images
|
||||
self.strength = strength
|
||||
self.seed = seed
|
||||
self.steps = steps
|
||||
self.guidance_scale = guidance_scale
|
||||
self.response_format = response_format
|
||||
self.image = image_to_base64(image)
|
||||
|
||||
def build_payload(self) -> dict:
|
||||
payload = {
|
||||
"model": "Qwen/Qwen-Image-Edit",
|
||||
"prompt": self.prompt,
|
||||
"image": self.image,
|
||||
"negative_prompt": self.negative_prompt,
|
||||
"n": self.num_images,
|
||||
"strength": self.strength,
|
||||
"seed": self.seed,
|
||||
"steps": self.steps,
|
||||
"guidance_scale": self.guidance_scale,
|
||||
"response_format": self.response_format,
|
||||
}
|
||||
return self._remove_empty_fields(payload)
|
||||
|
||||
def field_required(self):
|
||||
return ["prompt", "image"]
|
||||
|
||||
def field_order(self):
|
||||
return [
|
||||
"model",
|
||||
"prompt",
|
||||
"image",
|
||||
"negative_prompt",
|
||||
"n",
|
||||
"strength",
|
||||
"seed",
|
||||
"steps",
|
||||
"guidance_scale",
|
||||
"response_format",
|
||||
]
|
||||
|
||||
@@ -11,6 +11,7 @@ class Step1xEdit(BaseRequest):
|
||||
|
||||
Step1X-Edit transforms your photos with simple instructions into stunning, professional-quality edits—rivaling top proprietary tools.
|
||||
"""
|
||||
API_PATH = "/v1/images/generations"
|
||||
prompt: str = Field(...,
|
||||
description="The prompt to generate an image from.")
|
||||
image: Tensor = Field(
|
||||
|
||||
+133
-19
@@ -1,5 +1,10 @@
|
||||
import os
|
||||
import json
|
||||
import configparser
|
||||
import torch
|
||||
import server
|
||||
from aiohttp import web
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
try:
|
||||
current_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
@@ -18,6 +23,61 @@ except Exception as e:
|
||||
config = None
|
||||
|
||||
|
||||
secrets_path = os.path.join(parent_dir, 'secrets.json')
|
||||
|
||||
|
||||
def load_secrets():
|
||||
if not os.path.exists(secrets_path):
|
||||
with open(secrets_path, 'w') as secrets_file:
|
||||
json.dump({}, secrets_file, indent=2)
|
||||
with open(secrets_path) as secrets_file:
|
||||
return json.load(secrets_file)
|
||||
|
||||
|
||||
def save_secrets(secrets):
|
||||
with open(secrets_path, 'w') as secrets_file:
|
||||
json.dump(secrets, secrets_file, indent=2)
|
||||
|
||||
|
||||
async def get_modelverse_secrets(_request):
|
||||
return web.json_response(load_secrets())
|
||||
|
||||
|
||||
async def set_modelverse_secret(request):
|
||||
try:
|
||||
data = await request.json()
|
||||
except Exception:
|
||||
return web.json_response({"error": "Invalid or empty request body"}, status=400)
|
||||
|
||||
key = data.get("key", "").strip()
|
||||
value = data.get("value", "")
|
||||
if not key:
|
||||
return web.json_response({"error": "Key cannot be empty"}, status=400)
|
||||
|
||||
secrets = load_secrets()
|
||||
secrets[key] = value
|
||||
save_secrets(secrets)
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
async def delete_modelverse_secret(request):
|
||||
key = request.match_info["key"]
|
||||
secrets = load_secrets()
|
||||
if key not in secrets:
|
||||
return web.json_response({"error": "Key not found"}, status=404)
|
||||
|
||||
del secrets[key]
|
||||
save_secrets(secrets)
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
if not getattr(server.PromptServer.instance, "_modelverse_secrets_registered", False):
|
||||
server.PromptServer.instance.routes.get("/modelverse-secrets")(get_modelverse_secrets)
|
||||
server.PromptServer.instance.routes.post("/modelverse-secrets")(set_modelverse_secret)
|
||||
server.PromptServer.instance.routes.delete("/modelverse-secrets/{key}")(delete_modelverse_secret)
|
||||
server.PromptServer.instance._modelverse_secrets_registered = True
|
||||
|
||||
|
||||
class ModelverseAPIClient:
|
||||
"""
|
||||
Ucloud Modelverse API Client Node
|
||||
@@ -32,7 +92,11 @@ class ModelverseAPIClient:
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"api_key": ("STRING", {"multiline": False, "default": ""})
|
||||
"api_key": ("STRING", {
|
||||
"multiline": False,
|
||||
"default": "",
|
||||
"tooltip": "ModelVerse API key. Leave empty to read MODELVERSE_API_KEY from config.ini and avoid storing the key in workflow files."
|
||||
})
|
||||
},
|
||||
}
|
||||
|
||||
@@ -71,36 +135,82 @@ class ModelverseAPIClient:
|
||||
},)
|
||||
|
||||
|
||||
class ModelverseImagePacker:
|
||||
class ModelverseSecretClient:
|
||||
"""
|
||||
Ucloud Modelverse Image Packer
|
||||
UCloud Modelverse API Client from a locally managed secret.
|
||||
|
||||
This node packs multiple images into a list for multi-image editing.
|
||||
|
||||
Args:
|
||||
images1: The first image/image_list to be packed together with.
|
||||
images2: The second image/image_list to be packed together with. et cetera.
|
||||
|
||||
Returns:
|
||||
image_list: the pack of all input images.
|
||||
This node stores only the secret name in the workflow. The actual API key is
|
||||
read from secrets.json at execution time.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"images1": ("IMAGE,IMAGE_LIST", {"tooltip": "The first image/list to be packed together. Add more if you need."})
|
||||
"secret": ("STRING", {
|
||||
"multiline": False,
|
||||
"default": "",
|
||||
"tooltip": "Secret name from Modelverse Secrets Manager. The workflow stores this name, not the API key."
|
||||
})
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("MODELVERSE_API_CLIENT",)
|
||||
RETURN_NAMES = ("client",)
|
||||
|
||||
FUNCTION = "create_client"
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
|
||||
@classmethod
|
||||
def IS_CHANGED(cls, secret):
|
||||
return load_secrets().get(secret, "")
|
||||
|
||||
def create_client(self, secret):
|
||||
secret = secret.strip() if isinstance(secret, str) else secret
|
||||
if not secret:
|
||||
raise ValueError("Secret name is required")
|
||||
|
||||
secrets = load_secrets()
|
||||
api_key = secrets.get(secret)
|
||||
if not api_key:
|
||||
raise ValueError(f"Secret '{secret}' not found in secrets.json")
|
||||
|
||||
return ({
|
||||
"api_key": api_key
|
||||
},)
|
||||
|
||||
|
||||
class ModelverseImagePacker:
|
||||
"""
|
||||
Ucloud Modelverse Image Packer
|
||||
|
||||
This node packs multiple images into a batched IMAGE tensor for multi-image editing.
|
||||
|
||||
Args:
|
||||
images1: The first image to be packed together with.
|
||||
images2: The second image to be packed together with, et cetera.
|
||||
|
||||
Returns:
|
||||
images: batched IMAGE tensor for Flux Kontext Pro/Max multi-image mode.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"images1": (IO.IMAGE, {"tooltip": "The first image/list to be packed together. Add more if you need."})
|
||||
},
|
||||
"optional": {
|
||||
"images2": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The second image/list to be packed together."}),
|
||||
"images3": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The third image/list to be packed together."}),
|
||||
"images4": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The fourth image/list to be packed together."}),
|
||||
"images5": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The fifth image/list to be packed together."})
|
||||
"images2": (IO.IMAGE, {"default": None, "tooltip": "The second image/list to be packed together."}),
|
||||
"images3": (IO.IMAGE, {"default": None, "tooltip": "The third image/list to be packed together."}),
|
||||
"images4": (IO.IMAGE, {"default": None, "tooltip": "The fourth image/list to be packed together."}),
|
||||
"images5": (IO.IMAGE, {"default": None, "tooltip": "The fifth image/list to be packed together."})
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE_LIST",)
|
||||
RETURN_NAMES = ("image_list",)
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("images",)
|
||||
|
||||
FUNCTION = "pack_images"
|
||||
|
||||
@@ -121,14 +231,18 @@ class ModelverseImagePacker:
|
||||
for i in (images1, images2, images3, images4, images5):
|
||||
if i is not None:
|
||||
result.extend(to_list(i))
|
||||
return (result,)
|
||||
if not result:
|
||||
raise ValueError("At least one image is required")
|
||||
return (torch.cat(result, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
'UCloud ModelVerse Client': ModelverseAPIClient,
|
||||
'UCloud ModelVerse Secret Client': ModelverseSecretClient,
|
||||
'ModelVerse Image Packer': ModelverseImagePacker
|
||||
}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
'UCloud ModelVerse Client': 'Modelverse Client',
|
||||
'UCloud ModelVerse Secret Client': 'Modelverse Secret Client',
|
||||
'ModelVerse Image Packer': 'Modelverse Image Packer'
|
||||
}
|
||||
|
||||
+503
@@ -0,0 +1,503 @@
|
||||
import io
|
||||
import os
|
||||
import json
|
||||
import openai
|
||||
from openai import OpenAI
|
||||
import base64
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from typing import Optional, List, Dict, Any
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
from server import PromptServer
|
||||
import folder_paths
|
||||
|
||||
# Hardcoded models list
|
||||
DEFAULT_MODELS = [
|
||||
'ByteDance/doubao-seed-1.6',
|
||||
'Qwen/QwQ-32B',
|
||||
'Qwen/Qwen-Image',
|
||||
'Qwen/Qwen-Image-Edit',
|
||||
'Qwen/Qwen3-32B',
|
||||
'Qwen/Qwen3-Coder',
|
||||
'doubao-seedance-2-0-260128',
|
||||
'kling-v3',
|
||||
'kling-v3-omni',
|
||||
'happyhorse-1.0-t2v',
|
||||
'happyhorse-1.0-i2v',
|
||||
'happyhorse-1.0-r2v',
|
||||
'veo-3.1-generate-001',
|
||||
'veo-3.1-fast-generate-001',
|
||||
'baidu/ernie-4.5-turbo-128k',
|
||||
'baidu/ernie-4.5-turbo-vl-32k',
|
||||
'baidu/ernie-x1-turbo-32k',
|
||||
'black-forest-labs/flux-kontext-max',
|
||||
'black-forest-labs/flux-kontext-max/multi',
|
||||
'black-forest-labs/flux-kontext-max/text-to-image',
|
||||
'black-forest-labs/flux-kontext-pro',
|
||||
'black-forest-labs/flux-kontext-pro/multi',
|
||||
'black-forest-labs/flux-kontext-pro/text-to-image',
|
||||
'black-forest-labs/flux.1-dev',
|
||||
'claude-4-sonnet',
|
||||
'deepseek-ai/DeepSeek-R1',
|
||||
'deepseek-ai/DeepSeek-R1-0528',
|
||||
'deepseek-ai/DeepSeek-V3-0324',
|
||||
'deepseek-ai/DeepSeek-V3.1',
|
||||
'gemini-2.5-flash',
|
||||
'gemini-2.5-flash-image',
|
||||
'gemini-2.5-pro',
|
||||
'gemini-3.1-flash-image',
|
||||
'gemini-3-pro-image',
|
||||
'gpt-4.1-mini',
|
||||
'grok-4',
|
||||
'moonshotai/Kimi-K2-Instruct',
|
||||
'openai/gpt-4.1',
|
||||
'openai/gpt-5',
|
||||
'openai/gpt-5-mini',
|
||||
'viduq2',
|
||||
'viduq2-pro',
|
||||
'viduq2-turbo',
|
||||
'viduq2-pro-fast',
|
||||
'viduq3-pro',
|
||||
'viduq3-turbo',
|
||||
'zai-org/glm-5',
|
||||
'stepfun-ai/step1x-edit'
|
||||
]
|
||||
|
||||
# Default selected model
|
||||
DEFAULT_MODEL = "zai-org/glm-5"
|
||||
|
||||
|
||||
class ModelverseChat:
|
||||
"""OpenAI Chat node with support for text, images, and files"""
|
||||
|
||||
# Class-level cache for models list
|
||||
_cached_models = None
|
||||
_cache_initialized = False
|
||||
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def get_models_list(cls):
|
||||
"""Try to fetch models from ModelVerse API, fallback to default models if failed"""
|
||||
# Return cached models if already fetched
|
||||
if cls._cache_initialized:
|
||||
return cls._cached_models
|
||||
|
||||
try:
|
||||
# Try to create a client and fetch models
|
||||
# Note: This will work if there's a valid API key available, otherwise fallback
|
||||
client = OpenAI(base_url="https://api.modelverse.cn/v1",api_key="xxxsu")
|
||||
models_response = client.models.list()
|
||||
|
||||
# Extract model IDs from the response
|
||||
if hasattr(models_response, 'data') and models_response.data:
|
||||
model_ids = []
|
||||
for model in models_response.data:
|
||||
if hasattr(model, 'id') and model.id:
|
||||
model_ids.append(model.id)
|
||||
|
||||
if model_ids:
|
||||
print(f"ModelverseChat: Successfully fetched {len(model_ids)} models from API")
|
||||
cls._cached_models = sorted(model_ids) # Sort for better user experience
|
||||
cls._cache_initialized = True
|
||||
return cls._cached_models
|
||||
|
||||
# If no models found, fall back to default
|
||||
print("ModelverseChat: No models found in API response, using default models")
|
||||
cls._cached_models = DEFAULT_MODELS
|
||||
cls._cache_initialized = True
|
||||
return cls._cached_models
|
||||
|
||||
except Exception as e:
|
||||
print(f"ModelverseChat: Failed to fetch models from API ({str(e)}), using default models")
|
||||
cls._cached_models = DEFAULT_MODELS
|
||||
cls._cache_initialized = True
|
||||
return cls._cached_models
|
||||
|
||||
@classmethod
|
||||
def clear_models_cache(cls):
|
||||
"""Clear the cached models list to force a refresh on next call"""
|
||||
cls._cached_models = None
|
||||
cls._cache_initialized = False
|
||||
print("ModelverseChat: Models cache cleared")
|
||||
|
||||
def display_message_on_node(self, message: str, node_id: str) -> None:
|
||||
"""Display the current response message on the node UI."""
|
||||
render_spec = {
|
||||
"node_id": node_id,
|
||||
"component": "MessageDisplayWidget",
|
||||
"props": {
|
||||
"message": message,
|
||||
},
|
||||
}
|
||||
PromptServer.instance.send_sync(
|
||||
"display_component",
|
||||
render_spec,
|
||||
)
|
||||
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
# Get models list dynamically
|
||||
available_models = cls.get_models_list()
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (available_models, {"default": DEFAULT_MODEL if DEFAULT_MODEL in available_models else available_models[0] if available_models else DEFAULT_MODEL}),
|
||||
"user_prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "What can you tell me about this?"
|
||||
}),
|
||||
"temperature": (IO.FLOAT, {
|
||||
"default": 0.7,
|
||||
"min": 0.0,
|
||||
"max": 2.0,
|
||||
"step": 0.1
|
||||
}),
|
||||
"max_tokens": (IO.INT, {
|
||||
"default": 6000,
|
||||
"min": 1,
|
||||
"max": 128000
|
||||
}),
|
||||
"top_p": (IO.FLOAT, {
|
||||
"default": 1.0,
|
||||
"min": 0.0,
|
||||
"max": 1.0,
|
||||
"step": 0.01
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"system_prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "You are a helpful assistant."
|
||||
}),
|
||||
"image_in": (IO.IMAGE, {}),
|
||||
"files": ("OPENAI_INPUT_FILES", {
|
||||
"default": None,
|
||||
"tooltip": "Optional file(s) to use as context for the model. Accepts inputs from the OpenAI Input Files node."
|
||||
}),
|
||||
# "response_format": (["text", "json_object"], {"default": "text"}),
|
||||
"response_format": (["text"], {"default": "text"}),
|
||||
"presence_penalty": (IO.FLOAT, {
|
||||
"default": 0.0,
|
||||
"min": -2.0,
|
||||
"max": 2.0,
|
||||
"step": 0.1
|
||||
}),
|
||||
"frequency_penalty": (IO.FLOAT, {
|
||||
"default": 0.0,
|
||||
"min": -2.0,
|
||||
"max": 2.0,
|
||||
"step": 0.1
|
||||
}),
|
||||
},
|
||||
"hidden": {
|
||||
"unique_id": "UNIQUE_ID",
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING,)
|
||||
RETURN_NAMES = ("response",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
FUNCTION = "chat"
|
||||
|
||||
@classmethod
|
||||
def IS_CHANGED(cls, **kwargs):
|
||||
"""Tell ComfyUI that this node's inputs may change"""
|
||||
return "static"
|
||||
|
||||
def chat(self,
|
||||
client: Dict[str, str],
|
||||
model: str,
|
||||
user_prompt: str,
|
||||
temperature: float,
|
||||
max_tokens: int,
|
||||
top_p: float,
|
||||
unique_id: Optional[str] = None,
|
||||
system_prompt: Optional[str] = "You are a helpful assistant.",
|
||||
image_in: Optional[Any] = None,
|
||||
files: Optional[List[Any]] = None,
|
||||
response_format: str = "text",
|
||||
presence_penalty: float = 0.0,
|
||||
frequency_penalty: float = 0.0) -> tuple:
|
||||
|
||||
# Create ModelverseClient and get API key
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("No API key found in the client")
|
||||
|
||||
# Create ModelverseClient instance to get the actual API key
|
||||
modelverse_client = ModelverseClient(api_key)
|
||||
|
||||
# Initialize OpenAI client with the API key from ModelverseClient
|
||||
openai_client = openai.OpenAI(api_key=modelverse_client.api_key,base_url="https://api.modelverse.cn/v1")
|
||||
|
||||
# Build messages
|
||||
messages = []
|
||||
|
||||
# Add system message if provided
|
||||
if system_prompt:
|
||||
messages.append({
|
||||
"role": "system",
|
||||
"content": system_prompt
|
||||
})
|
||||
|
||||
# Build user message content
|
||||
user_content = []
|
||||
|
||||
# Add main prompt
|
||||
user_content.append({
|
||||
"type": "text",
|
||||
"text": user_prompt
|
||||
})
|
||||
|
||||
# Add file content if provided
|
||||
if files:
|
||||
print(f"OpenAIChat: Processing {len(files)} files")
|
||||
for file_info in files:
|
||||
# Handle the actual file format from OpenAIInputFiles
|
||||
if hasattr(file_info, 'file_data') and hasattr(file_info, 'filename'):
|
||||
# Decode base64 content
|
||||
try:
|
||||
# Extract base64 data (remove data:text/plain;base64, prefix)
|
||||
if file_info.file_data.startswith('data:'):
|
||||
base64_data = file_info.file_data.split(',', 1)[1]
|
||||
else:
|
||||
base64_data = file_info.file_data
|
||||
|
||||
# Decode base64 to text
|
||||
content = base64.b64decode(base64_data).decode('utf-8')
|
||||
|
||||
file_text = f"\n\nFile: {file_info.filename}\nContent:\n{content}"
|
||||
user_content.append({
|
||||
"type": "text",
|
||||
"text": file_text
|
||||
})
|
||||
print(f"OpenAIChat: Added file {file_info.filename} with {len(content)} characters")
|
||||
except Exception as e:
|
||||
print(f"OpenAIChat: Error decoding file {file_info.filename}: {str(e)}")
|
||||
elif isinstance(file_info, dict) and "content" in file_info and "filename" in file_info:
|
||||
# Handle simple dict format (backup)
|
||||
file_text = f"\n\nFile: {file_info['filename']}\nContent:\n{file_info['content']}"
|
||||
user_content.append({
|
||||
"type": "text",
|
||||
"text": file_text
|
||||
})
|
||||
print(f"OpenAIChat: Added file {file_info['filename']} with {len(file_info['content'])} characters")
|
||||
else:
|
||||
print(f"OpenAIChat: Invalid file format: {file_info}")
|
||||
else:
|
||||
print("OpenAIChat: No files provided")
|
||||
|
||||
# Add image if provided
|
||||
if image_in is not None:
|
||||
# Convert tensor to PIL Image
|
||||
# Handle both single images and batches
|
||||
if len(image_in.shape) == 4:
|
||||
# Take first image from batch
|
||||
image_in = image_in[0]
|
||||
|
||||
# Convert from tensor format (H, W, C) to numpy array
|
||||
image_array = image_in.cpu().numpy()
|
||||
|
||||
# Ensure values are in 0-255 range
|
||||
image_array = np.clip(255. * image_array, 0, 255).astype(np.uint8)
|
||||
|
||||
# Create PIL Image
|
||||
pil_image = Image.fromarray(image_array)
|
||||
|
||||
# Convert PIL Image to base64
|
||||
buffered = io.BytesIO()
|
||||
pil_image.save(buffered, format="PNG")
|
||||
img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
|
||||
|
||||
# Add image to content
|
||||
user_content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": f"data:image/png;base64,{img_str}"
|
||||
}
|
||||
})
|
||||
|
||||
# Add user message with all content
|
||||
if len(user_content) == 1:
|
||||
# If only text, use simple format
|
||||
messages.append({
|
||||
"role": "user",
|
||||
"content": user_content[0]["text"]
|
||||
})
|
||||
else:
|
||||
# If multiple content types, use array format
|
||||
messages.append({
|
||||
"role": "user",
|
||||
"content": user_content
|
||||
})
|
||||
|
||||
# Prepare API call parameters
|
||||
api_params = {
|
||||
"model": model,
|
||||
"messages": messages,
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
"top_p": top_p,
|
||||
"presence_penalty": presence_penalty,
|
||||
"frequency_penalty": frequency_penalty,
|
||||
}
|
||||
|
||||
# Add response format if JSON is requested
|
||||
if response_format == "json_object":
|
||||
api_params["response_format"] = {"type": "json_object"}
|
||||
|
||||
try:
|
||||
# Make API call to OpenAI
|
||||
response = openai_client.chat.completions.create(**api_params)
|
||||
|
||||
# Extract response content
|
||||
if response.choices and len(response.choices) > 0:
|
||||
content = response.choices[0].message.content
|
||||
if content is None:
|
||||
raise ValueError("No content in response")
|
||||
|
||||
# Display the response message on the node UI
|
||||
if unique_id:
|
||||
self.display_message_on_node(content.strip(), unique_id)
|
||||
|
||||
# Return the response as a tuple
|
||||
return (content.strip(),)
|
||||
else:
|
||||
raise ValueError("No choices in response")
|
||||
|
||||
except Exception as e:
|
||||
error_msg = f"OpenAI API Error: {str(e)}"
|
||||
print(error_msg)
|
||||
return (error_msg,)
|
||||
|
||||
|
||||
class ModelverseInputFiles:
|
||||
"""
|
||||
Loads and formats input files for OpenAI API.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
"""
|
||||
For details about the supported file input types, see:
|
||||
https://platform.openai.com/docs/guides/pdf-files?api-mode=responses
|
||||
"""
|
||||
input_dir = folder_paths.get_input_directory()
|
||||
input_files = [
|
||||
f
|
||||
for f in os.scandir(input_dir)
|
||||
if f.is_file()
|
||||
and (f.name.endswith(".txt") or f.name.endswith(".pdf") or f.name.endswith(".md"))
|
||||
and f.stat().st_size < 32 * 1024 * 1024
|
||||
]
|
||||
input_files = sorted(input_files, key=lambda x: x.name)
|
||||
input_files = [f.name for f in input_files]
|
||||
return {
|
||||
"required": {},
|
||||
"optional": {
|
||||
"file": (
|
||||
IO.COMBO,
|
||||
{
|
||||
"options": input_files,
|
||||
"default": input_files[0] if input_files else None,
|
||||
"tooltip": "Input files to include as context for the model. Only accepts text (.txt), markdown (.md) and PDF (.pdf) files for now.",
|
||||
},
|
||||
),
|
||||
"OPENAI_INPUT_FILES": (
|
||||
"OPENAI_INPUT_FILES",
|
||||
{
|
||||
"tooltip": "An optional additional file(s) to batch together with the file loaded from this node. Allows chaining of input files so that a single message can include multiple input files.",
|
||||
"default": None,
|
||||
},
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
DESCRIPTION = "Loads and prepares input files (text, markdown, pdf, etc.) to include as inputs for the OpenAI Chat Node. The files will be read by the OpenAI model when generating a response. 🛈 TIP: Can be chained together with other OpenAI Input File nodes."
|
||||
RETURN_TYPES = ("OPENAI_INPUT_FILES",)
|
||||
FUNCTION = "prepare_files"
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
|
||||
def read_file_content(self, file_path: str) -> str:
|
||||
"""Read content from a file."""
|
||||
try:
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
return f.read()
|
||||
except UnicodeDecodeError:
|
||||
# Try with different encoding if UTF-8 fails
|
||||
try:
|
||||
with open(file_path, 'r', encoding='gbk') as f:
|
||||
return f.read()
|
||||
except:
|
||||
return f"Error: Unable to read file {file_path} - encoding issue"
|
||||
except Exception as e:
|
||||
return f"Error reading file {file_path}: {str(e)}"
|
||||
|
||||
def text_to_data_uri(self, content: str) -> str:
|
||||
"""Convert text content to data URI."""
|
||||
encoded_content = base64.b64encode(content.encode('utf-8')).decode('ascii')
|
||||
return f"data:text/plain;base64,{encoded_content}"
|
||||
|
||||
def create_input_file_content(self, file_path: str):
|
||||
"""Create a file content object compatible with the reference implementation."""
|
||||
content = self.read_file_content(file_path)
|
||||
|
||||
# Create a simple object that mimics the expected structure
|
||||
class InputFileContent:
|
||||
def __init__(self, content, filename):
|
||||
self.file_data = self.text_to_data_uri(content)
|
||||
self.filename = filename
|
||||
self.type = "input_file"
|
||||
self.file_id = None
|
||||
|
||||
def text_to_data_uri(self, content: str) -> str:
|
||||
encoded_content = base64.b64encode(content.encode('utf-8')).decode('ascii')
|
||||
return f"data:text/plain;base64,{encoded_content}"
|
||||
|
||||
return InputFileContent(content, os.path.basename(file_path))
|
||||
|
||||
def prepare_files(
|
||||
self, file: str = None, OPENAI_INPUT_FILES: List[Any] = None
|
||||
) -> tuple:
|
||||
"""
|
||||
Loads and formats input files for OpenAI API.
|
||||
"""
|
||||
files = []
|
||||
|
||||
# Add current file if provided
|
||||
if file is not None and file != "":
|
||||
try:
|
||||
file_path = folder_paths.get_annotated_filepath(file)
|
||||
input_file_content = self.create_input_file_content(file_path)
|
||||
files.append(input_file_content)
|
||||
print(f"OpenAIInputFiles: Successfully loaded file {file}")
|
||||
except Exception as e:
|
||||
print(f"OpenAIInputFiles: Error loading file {file}: {str(e)}")
|
||||
|
||||
# Add previous files if provided
|
||||
if OPENAI_INPUT_FILES is not None:
|
||||
files.extend(OPENAI_INPUT_FILES)
|
||||
print(f"OpenAIInputFiles: Added {len(OPENAI_INPUT_FILES)} previous files")
|
||||
|
||||
print(f"OpenAIInputFiles: Returning {len(files)} files total")
|
||||
return (files,)
|
||||
|
||||
|
||||
|
||||
# Node registration
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"ModelverseChat": ModelverseChat,
|
||||
"ModelverseInputFiles": ModelverseInputFiles,
|
||||
# "OpenAICaptionImage": OpenAICaptionImage,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"ModelverseChat": "Modelverse Chat",
|
||||
"ModelverseInputFiles": "Modelverse Input Files",
|
||||
# "OpenAICaptionImage": "OpenAI Caption Image",
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
import os
|
||||
import re
|
||||
import requests
|
||||
import folder_paths
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
from comfy_api_nodes.util import download_url_to_video_output
|
||||
|
||||
|
||||
class ModelversePreviewVideo:
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"video_url": (IO.STRING, {"forceInput": True}),
|
||||
"filename_prefix": (IO.STRING, {"default": "Modelverse"}),
|
||||
"save_output": (IO.BOOLEAN, {"default": True}),
|
||||
}
|
||||
}
|
||||
|
||||
OUTPUT_NODE = True
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
|
||||
RETURN_TYPES = (IO.VIDEO,)
|
||||
RETURN_NAMES = ("video",)
|
||||
|
||||
async def run(self, video_url, filename_prefix, save_output):
|
||||
if type(video_url) == list:
|
||||
video_url = video_url[0]
|
||||
|
||||
# Save to file if requested (do this first before download_url_to_video_output consumes the data)
|
||||
if save_output:
|
||||
output_dir = folder_paths.get_output_directory()
|
||||
(
|
||||
full_output_folder,
|
||||
filename,
|
||||
_,
|
||||
_,
|
||||
_,
|
||||
) = folder_paths.get_save_image_path(filename_prefix, output_dir)
|
||||
|
||||
max_counter = 0
|
||||
matcher = re.compile(f"{re.escape(filename)}_(\\d+)\\D*\\..+", re.IGNORECASE)
|
||||
for existing_file in os.listdir(full_output_folder):
|
||||
match = matcher.fullmatch(existing_file)
|
||||
if match:
|
||||
file_counter = int(match.group(1))
|
||||
if file_counter > max_counter:
|
||||
max_counter = file_counter
|
||||
|
||||
counter = max_counter + 1
|
||||
file = f"{filename}_{counter:05}.mp4"
|
||||
file_path = os.path.join(full_output_folder, file)
|
||||
|
||||
response = requests.get(video_url, timeout=120)
|
||||
response.raise_for_status()
|
||||
with open(file_path, "wb") as f:
|
||||
f.write(response.content)
|
||||
|
||||
# Download video and get VIDEO output
|
||||
video = await download_url_to_video_output(video_url)
|
||||
|
||||
return {"ui": {"video_url": [video_url]}, "result": (video,)}
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse_PreviewVideo": ModelversePreviewVideo,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse_PreviewVideo": "Modelverse Preview Video",
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
import base64
|
||||
import torch
|
||||
from typing import Optional, List
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.qwen_image_edit import QwenImageEdit
|
||||
|
||||
|
||||
class QwenImageEditNode:
|
||||
"""
|
||||
Qwen/Qwen-Image-Edit image editing via Modelverse /v1/images/generations API.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "Convert to quick pencil sketch"}),
|
||||
"image": (IO.IMAGE, {"tooltip": "Input image to edit"}),
|
||||
"negative_prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Negative prompt"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
|
||||
"strength": (IO.FLOAT, {"default": 0.8, "min": 0.0, "max": 1.0, "step": 0.01, "display": "number"}),
|
||||
"steps": (IO.INT, {"default": 20, "min": 1, "max": 50, "step": 1, "display": "number"}),
|
||||
"seed": (IO.INT, {"default": -1, "min": -1, "max": 0xffffffffffffffff, "control_after_generate": True}),
|
||||
"guidance_scale": (IO.FLOAT, {"default": 2.5, "min": 1.0, "max": 10.0, "step": 0.1, "display": "number"}),
|
||||
"response_format": (["url", "b64_json"], {"default": "url"}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Qwen-Image"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(
|
||||
self,
|
||||
client,
|
||||
prompt: str,
|
||||
image,
|
||||
negative_prompt: str = "",
|
||||
num_requests: int = 1,
|
||||
num_images: int = 1,
|
||||
strength: float = 0.8,
|
||||
steps: int = 20,
|
||||
seed: int = -1,
|
||||
guidance_scale: float = 2.5,
|
||||
response_format: str = "url",
|
||||
):
|
||||
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
if image is None:
|
||||
raise ValueError("Input image is required")
|
||||
|
||||
print("INFO:", "Running Qwen/Qwen-Image-Edit.")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
tasks = [
|
||||
mv_client.async_send_request(
|
||||
QwenImageEdit(
|
||||
prompt=prompt,
|
||||
image=image,
|
||||
negative_prompt=negative_prompt,
|
||||
num_images=num_images,
|
||||
strength=strength,
|
||||
seed=seed + i,
|
||||
steps=steps,
|
||||
guidance_scale=guidance_scale,
|
||||
response_format=response_format,
|
||||
)
|
||||
)
|
||||
for i in range(num_requests)
|
||||
]
|
||||
|
||||
results = await mv_client.run_tasks(tasks) # list of data lists
|
||||
|
||||
output_images_list: List[torch.Tensor] = []
|
||||
for data_list in results:
|
||||
if not data_list:
|
||||
print("WARN:", "No output in current request. Skipping...")
|
||||
continue
|
||||
|
||||
if response_format == "url":
|
||||
output_images = imageurl2tensor(data_list)
|
||||
else:
|
||||
# b64_json path
|
||||
images = []
|
||||
for item in data_list:
|
||||
b64v = item.get("b64_json") or item.get("b64")
|
||||
if not b64v:
|
||||
continue
|
||||
if isinstance(b64v, str) and b64v.startswith("data:"):
|
||||
try:
|
||||
b64v = b64v.split(",", 1)[1]
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
img_bytes = base64.b64decode(b64v)
|
||||
pil_img = decode_image(img_bytes)
|
||||
images.append(pil_img)
|
||||
except Exception:
|
||||
continue
|
||||
if not images:
|
||||
print("WARN:", "No decodable base64 image found.")
|
||||
continue
|
||||
output_images = images2tensor(images)
|
||||
|
||||
output_images_list.append(output_images)
|
||||
|
||||
if not output_images_list:
|
||||
return (torch.zeros((1, 3, 1, 1)),)
|
||||
|
||||
return (torch.cat(output_images_list, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse QwenImageEditNode": QwenImageEditNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse QwenImageEditNode": "Modelverse Qwen Image Edit",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
import base64
|
||||
import torch
|
||||
from typing import List
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.qwen_image import QwenImage
|
||||
|
||||
|
||||
class QwenImageT2INode:
|
||||
"""
|
||||
Qwen/Qwen-Image text-to-image via Modelverse /v1/images/generations API.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower"}),
|
||||
"aspect_ratio": (["21:9", "16:9", "4:3", "3:2", "1:1", "2:3", "3:4", "9:16", "9:21"], {"default": "1:1"}),
|
||||
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
|
||||
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
|
||||
"steps": (IO.INT, {"default": 20, "min": 1, "max": 50, "step": 1, "display": "number"}),
|
||||
"seed": (IO.INT, {"default": -1, "min": -1, "max": 0xffffffffffffffff, "control_after_generate": True}),
|
||||
"guidance_scale": (IO.FLOAT, {"default": 2.5, "min": 1.0, "max": 10.0, "step": 0.1, "display": "number"}),
|
||||
"response_format": (["url", "b64_json"], {"default": "url"}),
|
||||
},
|
||||
"optional": {
|
||||
"negative_prompt": (IO.STRING, {"multiline": True, "default": ""}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.IMAGE,)
|
||||
RETURN_NAMES = ("image",)
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Qwen-Image"
|
||||
FUNCTION = "execute"
|
||||
|
||||
async def execute(
|
||||
self,
|
||||
client,
|
||||
prompt: str,
|
||||
aspect_ratio: str = "1:1",
|
||||
num_requests: int = 1,
|
||||
num_images: int = 1,
|
||||
steps: int = 20,
|
||||
seed: int = -1,
|
||||
guidance_scale: float = 2.5,
|
||||
response_format: str = "url",
|
||||
negative_prompt: str = "",
|
||||
):
|
||||
|
||||
if not prompt:
|
||||
raise ValueError("Prompt is required")
|
||||
|
||||
mv_client = ModelverseClient(client["api_key"])
|
||||
|
||||
tasks = [
|
||||
mv_client.async_send_request(
|
||||
QwenImage(
|
||||
prompt=prompt,
|
||||
aspect_ratio=aspect_ratio,
|
||||
num_images=num_images,
|
||||
seed=seed + i,
|
||||
steps=steps,
|
||||
guidance_scale=guidance_scale,
|
||||
negative_prompt=negative_prompt,
|
||||
response_format=response_format,
|
||||
)
|
||||
)
|
||||
for i in range(num_requests)
|
||||
]
|
||||
|
||||
results = await mv_client.run_tasks(tasks)
|
||||
|
||||
output_images_list: List[torch.Tensor] = []
|
||||
for data_list in results:
|
||||
if not data_list:
|
||||
print("WARN:", "No output in current request. Skipping...")
|
||||
continue
|
||||
|
||||
if response_format == "url":
|
||||
output_images = imageurl2tensor(data_list)
|
||||
else:
|
||||
images = []
|
||||
for item in data_list:
|
||||
b64v = item.get("b64_json") or item.get("b64")
|
||||
if not b64v:
|
||||
continue
|
||||
if isinstance(b64v, str) and b64v.startswith("data:"):
|
||||
try:
|
||||
b64v = b64v.split(",", 1)[1]
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
img_bytes = base64.b64decode(b64v)
|
||||
pil_img = decode_image(img_bytes)
|
||||
images.append(pil_img)
|
||||
except Exception:
|
||||
continue
|
||||
if not images:
|
||||
print("WARN:", "No decodable base64 image found.")
|
||||
continue
|
||||
output_images = images2tensor(images)
|
||||
|
||||
output_images_list.append(output_images)
|
||||
|
||||
if not output_images_list:
|
||||
return (torch.zeros((1, 3, 1, 1)),)
|
||||
|
||||
return (torch.cat(output_images_list, dim=0),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse QwenImageT2INode": QwenImageT2INode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse QwenImageT2INode": "Modelverse Qwen Image",
|
||||
}
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
"""
|
||||
OpenAI Sora2 Img2Video - 图生视频模型
|
||||
Models: openai/sora-2/image-to-video, openai/sora-2/image-to-video-pro
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["openai/sora-2/image-to-video", "openai/sora-2/image-to-video-pro"]
|
||||
RESOLUTIONS_PRO = ["720p", "1080p"]
|
||||
DURATIONS = [4, 8, 12]
|
||||
|
||||
|
||||
class SoraImg2VideoNode:
|
||||
"""
|
||||
OpenAI Sora2 Img2Video - 图生视频
|
||||
Models: image-to-video (普通版), image-to-video-pro (Pro版,支持1080p)
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "openai/sora-2/image-to-video", "tooltip": "image-to-video: 普通版, image-to-video-pro: Pro版(支持1080p)"}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
|
||||
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL (与image二选一)"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "提示词,用于指导视频生成"}),
|
||||
"resolution": (RESOLUTIONS_PRO, {"default": "720p", "tooltip": "分辨率 (Pro版支持1080p)"}),
|
||||
"duration": (DURATIONS, {"default": 4, "tooltip": "视频时长(秒): 4, 8, 12"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Sora"
|
||||
|
||||
def generate(self, client, model, first_frame_image=None, first_frame_url="",
|
||||
prompt="", resolution="720p", duration=4):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
# Validate resolution for non-pro model
|
||||
if model == "openai/sora-2/image-to-video" and resolution == "1080p":
|
||||
raise ValueError("普通版不支持 1080p,请使用 Pro 版")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
# Validate first frame input
|
||||
has_url = first_frame_url and first_frame_url.strip()
|
||||
has_image = first_frame_image is not None
|
||||
|
||||
if has_url and has_image:
|
||||
raise ValueError("请提供 first_frame_image 或 first_frame_url,不能同时提供")
|
||||
if not has_url and not has_image:
|
||||
raise ValueError("必须提供 first_frame_image 或 first_frame_url")
|
||||
|
||||
task_input = {}
|
||||
if has_url:
|
||||
task_input["first_frame_url"] = first_frame_url.strip()
|
||||
else:
|
||||
task_input["first_frame_url"] = image_to_base64(first_frame_image)
|
||||
|
||||
if prompt and prompt.strip():
|
||||
task_input["prompt"] = prompt.strip()
|
||||
|
||||
parameters = {
|
||||
"duration": duration,
|
||||
}
|
||||
|
||||
# Pro版支持resolution参数
|
||||
if model == "openai/sora-2/image-to-video-pro":
|
||||
parameters["resolution"] = resolution
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Sora I2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Sora_Img2Video": SoraImg2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Sora_Img2Video": "Sora Img2Video",
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
"""
|
||||
OpenAI Sora2 Text2Video - 文生视频模型
|
||||
Models: openai/sora-2/text-to-video, openai/sora-2/text-to-video-pro
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["openai/sora-2/text-to-video", "openai/sora-2/text-to-video-pro"]
|
||||
SIZES = ["720x1280", "1280x720"]
|
||||
SIZES_PRO = ["720x1280", "1280x720", "1024x1792", "1792x1024"]
|
||||
DURATIONS = [4, 8, 12]
|
||||
|
||||
|
||||
class SoraText2VideoNode:
|
||||
"""
|
||||
OpenAI Sora2 Text2Video - 文生视频
|
||||
Models: text-to-video (普通版), text-to-video-pro (Pro版,支持更多分辨率)
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "openai/sora-2/text-to-video", "tooltip": "text-to-video: 普通版, text-to-video-pro: Pro版"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "A beautiful girl is dancing", "tooltip": "提示词,用于指导视频生成"}),
|
||||
},
|
||||
"optional": {
|
||||
"size": (SIZES_PRO, {"default": "720x1280", "tooltip": "视频尺寸 (Pro版支持更多选项)"}),
|
||||
"duration": (DURATIONS, {"default": 4, "tooltip": "视频时长(秒): 4, 8, 12"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Sora"
|
||||
|
||||
def generate(self, client, model, prompt, size="720x1280", duration=4):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
# Validate size for non-pro model
|
||||
if model == "openai/sora-2/text-to-video" and size not in SIZES:
|
||||
raise ValueError(f"普通版仅支持 {SIZES},请使用 Pro 版获取更多分辨率选项")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
task_input = {"prompt": prompt}
|
||||
parameters = {
|
||||
"size": size,
|
||||
"duration": duration,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Sora T2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Sora_Text2Video": SoraText2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Sora_Text2Video": "Sora Text2Video",
|
||||
}
|
||||
+9
-9
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.requests.step1x_edit import Step1xEdit
|
||||
import torch
|
||||
import asyncio
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class Step1xEditNode:
|
||||
@@ -17,10 +17,10 @@ class Step1xEditNode:
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "The prompt to guide the image edit."}),
|
||||
"negative_prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "The negative prompt to use."}),
|
||||
"image": ("IMAGE", {"default": None, "tooltip": "The image to be edited."}),
|
||||
"num_requests": ("INT", {
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "The prompt to guide the image edit."}),
|
||||
"negative_prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "The negative prompt to use."}),
|
||||
"image": (IO.IMAGE, {"default": None, "tooltip": "The image to be edited."}),
|
||||
"num_requests": (IO.INT, {
|
||||
"default": 1,
|
||||
"min": 1,
|
||||
"max": 10,
|
||||
@@ -28,14 +28,14 @@ class Step1xEditNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of request to make (1 to 10)"
|
||||
}),
|
||||
"seed": ("INT", {
|
||||
"seed": (IO.INT, {
|
||||
"default": -1,
|
||||
"min": -1,
|
||||
"max": 0xffffffffffffffff,
|
||||
"control_after_generate": True,
|
||||
"tooltip": "Random seed for reproducible results. -1 for random seed"
|
||||
}),
|
||||
"num_inference_steps": ("INT", {
|
||||
"num_inference_steps": (IO.INT, {
|
||||
"default": 30,
|
||||
"min": 1,
|
||||
"max": 50,
|
||||
@@ -43,7 +43,7 @@ class Step1xEditNode:
|
||||
"display": "number",
|
||||
"tooltip": "Number of inference steps (1 to 50)"
|
||||
}),
|
||||
"guidance_scale": ("FLOAT", {
|
||||
"guidance_scale": (IO.FLOAT, {
|
||||
"default": 4.0,
|
||||
"min": 0.0,
|
||||
"max": 20.0,
|
||||
@@ -54,7 +54,7 @@ class Step1xEditNode:
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", )
|
||||
RETURN_TYPES = (IO.IMAGE, )
|
||||
RETURN_NAMES = ("image", )
|
||||
|
||||
CATEGORY = "UCLOUD_MODELVERSE"
|
||||
|
||||
+211
@@ -0,0 +1,211 @@
|
||||
"""
|
||||
Google Veo 3.1 video generation
|
||||
Models: veo-3.1-generate-001, veo-3.1-fast-generate-001
|
||||
"""
|
||||
import base64
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import decode_image, encode_image, fetch_image, tensor2images
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["veo-3.1-generate-001", "veo-3.1-fast-generate-001"]
|
||||
ASPECT_RATIOS = ["16:9", "9:16"]
|
||||
RESOLUTIONS = ["720p", "1080p"]
|
||||
DURATIONS = [4, 6, 8]
|
||||
PERSON_GENERATIONS = ["dont_allow", "allow_adult"]
|
||||
|
||||
|
||||
def _bytes_to_veo_image(data_bytes, fmt):
|
||||
mime = "image/png" if fmt == "PNG" else "image/jpeg"
|
||||
return {
|
||||
"bytesBase64Encoded": base64.b64encode(data_bytes).decode("utf-8"),
|
||||
"mimeType": mime,
|
||||
}
|
||||
|
||||
|
||||
def _tensor_to_veo_image(tensor):
|
||||
data_bytes, fmt = encode_image(tensor2images(tensor)[0])
|
||||
return _bytes_to_veo_image(data_bytes, fmt)
|
||||
|
||||
|
||||
def _url_to_veo_image(url, label):
|
||||
url = url.strip()
|
||||
if url.startswith("data:image/") and "," in url:
|
||||
header, data = url.split(",", 1)
|
||||
mime = header.split(";")[0].replace("data:", "")
|
||||
return {"bytesBase64Encoded": data, "mimeType": mime}
|
||||
if url.startswith(("http://", "https://")):
|
||||
image_data = fetch_image(url)
|
||||
img = decode_image(image_data)
|
||||
data_bytes, fmt = encode_image(img)
|
||||
return _bytes_to_veo_image(data_bytes, fmt)
|
||||
raise ValueError(f"{label}: URL must be http(s) or a data:image/...;base64,... value")
|
||||
|
||||
|
||||
def _resolve_veo_image(image, url, label):
|
||||
has_url = url and url.strip()
|
||||
has_image = image is not None
|
||||
if has_url and has_image:
|
||||
raise ValueError(f"{label}: provide either image or url, not both")
|
||||
if has_url:
|
||||
return _url_to_veo_image(url, label)
|
||||
if has_image:
|
||||
return _tensor_to_veo_image(image)
|
||||
return None
|
||||
|
||||
|
||||
class Veo31VideoNode:
|
||||
"""
|
||||
Veo 3.1 video generation.
|
||||
Supports text-to-video, image-to-video, and first/last-frame video.
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {
|
||||
"default": "veo-3.1-generate-001",
|
||||
"tooltip": "veo-3.1-generate-001: standard quality, veo-3.1-fast-generate-001: faster generation",
|
||||
}),
|
||||
"prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Text prompt describing the desired video",
|
||||
}),
|
||||
"generate_audio": (IO.BOOLEAN, {
|
||||
"default": True,
|
||||
"tooltip": "Whether to generate synchronized audio (required by Veo API)",
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image for image-to-video"}),
|
||||
"first_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "First frame image URL (use either this OR first_frame_image)",
|
||||
}),
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image for start-end video"}),
|
||||
"last_frame_url": (IO.STRING, {
|
||||
"default": "",
|
||||
"tooltip": "Last frame image URL (use either this OR last_frame_image)",
|
||||
}),
|
||||
"negative_prompt": (IO.STRING, {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": "Negative prompt for unwanted elements",
|
||||
}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {
|
||||
"default": "16:9",
|
||||
"tooltip": "Output aspect ratio",
|
||||
}),
|
||||
"resolution": (RESOLUTIONS, {
|
||||
"default": "720p",
|
||||
"tooltip": "Output resolution",
|
||||
}),
|
||||
"duration": (DURATIONS, {
|
||||
"default": 8,
|
||||
"tooltip": "Video duration in seconds: 4, 6, or 8",
|
||||
}),
|
||||
"seed": (IO.INT, {
|
||||
"default": 0, "min": 0, "max": 4294967295,
|
||||
"tooltip": "Random seed (0 to skip)",
|
||||
}),
|
||||
"person_generation": (PERSON_GENERATIONS, {
|
||||
"default": "allow_adult",
|
||||
"tooltip": "Safety setting for person/face generation",
|
||||
}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Veo"
|
||||
|
||||
def generate(
|
||||
self,
|
||||
client,
|
||||
model,
|
||||
prompt,
|
||||
generate_audio=True,
|
||||
first_frame_image=None,
|
||||
first_frame_url="",
|
||||
last_frame_image=None,
|
||||
last_frame_url="",
|
||||
negative_prompt="",
|
||||
aspect_ratio="16:9",
|
||||
resolution="720p",
|
||||
duration=8,
|
||||
seed=0,
|
||||
person_generation="allow_adult",
|
||||
):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
if not prompt or not prompt.strip():
|
||||
raise ValueError("prompt is required for Veo 3.1")
|
||||
|
||||
first_image = _resolve_veo_image(first_frame_image, first_frame_url, "First frame")
|
||||
last_image = _resolve_veo_image(last_frame_image, last_frame_url, "Last frame")
|
||||
if last_image and not first_image:
|
||||
raise ValueError("First frame is required when last frame is provided")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
task_input = {"prompt": prompt.strip()}
|
||||
if negative_prompt and negative_prompt.strip():
|
||||
task_input["negative_prompt"] = negative_prompt.strip()
|
||||
if first_image:
|
||||
task_input["image"] = first_image
|
||||
if last_image:
|
||||
task_input["last_frame"] = last_image
|
||||
|
||||
parameters = {
|
||||
"generate_audio": generate_audio,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
"resolution": resolution,
|
||||
"duration": duration,
|
||||
"person_generation": person_generation,
|
||||
}
|
||||
if seed > 0:
|
||||
parameters["seed"] = seed
|
||||
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Veo 3.1 task submitted: model={model}, task_id={task_id}")
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
if task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
if task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
continue
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Veo_3_1_Video": Veo31VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Veo_3_1_Video": "Modelverse Veo 3.1 Video",
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
"""
|
||||
Vidu Extend - 视频延长模型
|
||||
Models: viduq2-pro, viduq2-turbo
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["viduq2-pro", "viduq2-turbo"]
|
||||
RESOLUTIONS = ["540p", "720p", "1080p"]
|
||||
|
||||
|
||||
class ViduExtendNode:
|
||||
"""
|
||||
Vidu Extend - 视频延长
|
||||
Models: viduq2-pro, viduq2-turbo
|
||||
输入视频时长需4秒-1分钟,延长1-7秒
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "viduq2-turbo", "tooltip": "viduq2-pro: 效果好, viduq2-turbo: 生成快"}),
|
||||
"video_url": (IO.STRING, {"default": "", "tooltip": "需要延长的视频URL (4秒-1分钟)"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 1, "max": 7, "step": 1, "tooltip": "延长时长(秒), 1-7"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
|
||||
},
|
||||
"optional": {
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "延长到尾帧的参考图像"}),
|
||||
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "延长到尾帧的参考图像URL"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "延长提示词,控制延长内容"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
|
||||
|
||||
def generate(self, client, model, video_url, duration, resolution,
|
||||
last_frame_image=None, last_frame_url="", prompt=""):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
if not video_url or not video_url.strip():
|
||||
raise ValueError("必须提供 video_url")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
task_input = {
|
||||
"video_url": video_url.strip(),
|
||||
}
|
||||
|
||||
# Optional last frame
|
||||
has_last_url = last_frame_url and last_frame_url.strip()
|
||||
has_last_image = last_frame_image is not None
|
||||
if has_last_url and has_last_image:
|
||||
raise ValueError("尾帧:请提供 image 或 url,不能同时提供")
|
||||
|
||||
if has_last_url:
|
||||
task_input["last_frame_url"] = last_frame_url.strip()
|
||||
elif has_last_image:
|
||||
task_input["last_frame_url"] = image_to_base64(last_frame_image)
|
||||
|
||||
if prompt and prompt.strip():
|
||||
task_input["prompt"] = prompt.strip()
|
||||
|
||||
parameters = {
|
||||
"vidu_type": "extend",
|
||||
"duration": duration,
|
||||
"resolution": resolution,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Vidu Extend task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url_result = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url_result, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Vidu_Extend": ViduExtendNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Vidu_Extend": "Vidu Extend",
|
||||
}
|
||||
+124
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
Vidu Img2Video - 图生视频模型
|
||||
Models: viduq3-pro, viduq3-turbo, viduq2-pro, viduq2-turbo, viduq2-pro-fast
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2-pro", "viduq2-turbo", "viduq2-pro-fast"]
|
||||
RESOLUTIONS = ["540p", "720p", "1080p"]
|
||||
MOVEMENT_AMPLITUDES = ["auto", "small", "medium", "large"]
|
||||
|
||||
|
||||
class ViduImg2VideoNode:
|
||||
"""
|
||||
Vidu Img2Video - 图生视频
|
||||
Models: viduq3-pro/viduq3-turbo (1-16s), viduq2-pro/viduq2-turbo/viduq2-pro-fast (1-10s)
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro/turbo: 支持1-16秒; viduq2-pro/turbo/pro-fast: 支持1-10秒"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 1, "max": 16, "step": 1, "tooltip": "视频时长(秒),viduq3系列支持1-16秒,viduq2系列支持1-10秒"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
|
||||
"movement_amplitude": (MOVEMENT_AMPLITUDES, {"default": "auto", "tooltip": "运动幅度"}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
|
||||
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL (与image二选一)"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "文本提示词,最长2000字符"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
|
||||
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
|
||||
|
||||
def generate(self, client, model, duration, resolution, movement_amplitude,
|
||||
first_frame_image=None, first_frame_url="", prompt="", seed=0, bgm=False):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
# Validate first frame input
|
||||
has_url = first_frame_url and first_frame_url.strip()
|
||||
has_image = first_frame_image is not None
|
||||
|
||||
if has_url and has_image:
|
||||
raise ValueError("请提供 first_frame_image 或 first_frame_url,不能同时提供")
|
||||
if not has_url and not has_image:
|
||||
raise ValueError("必须提供 first_frame_image 或 first_frame_url")
|
||||
if duration > 10 and not model.startswith("viduq3-"):
|
||||
raise ValueError("只有 viduq3 系列模型支持超过10秒的视频时长")
|
||||
|
||||
task_input = {}
|
||||
if has_url:
|
||||
task_input["first_frame_url"] = first_frame_url.strip()
|
||||
else:
|
||||
task_input["first_frame_url"] = image_to_base64(first_frame_image)
|
||||
|
||||
if prompt and prompt.strip():
|
||||
task_input["prompt"] = prompt.strip()
|
||||
|
||||
parameters = {
|
||||
"vidu_type": "img2video",
|
||||
"duration": duration,
|
||||
"resolution": resolution,
|
||||
"movement_amplitude": movement_amplitude,
|
||||
"seed": seed,
|
||||
"bgm": bgm,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Vidu I2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Vidu_Img2Video": ViduImg2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Vidu_Img2Video": "Vidu Img2Video",
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
"""
|
||||
Vidu Reference2Video - 参考生视频模型
|
||||
Models: viduq3-turbo, viduq2
|
||||
支持1-7张参考图片,生成具备主体一致的视频
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["viduq3-turbo", "viduq2"]
|
||||
ASPECT_RATIOS = ["16:9", "9:16", "3:4", "4:3", "1:1"]
|
||||
RESOLUTIONS = ["540p", "720p", "1080p"]
|
||||
|
||||
|
||||
class ViduReference2VideoNode:
|
||||
"""
|
||||
Vidu Reference2Video - 参考生视频
|
||||
Models: viduq3-turbo, viduq2
|
||||
支持1-7张参考图片,生成具备主体一致的视频
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "viduq3-turbo", "tooltip": "viduq3-turbo: 生成快, viduq2: 旧版模型"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "make it dance", "tooltip": "文本提示词,最长2000字符"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 1, "max": 10, "step": 1, "tooltip": "视频时长(秒)"}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "长宽比"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
|
||||
},
|
||||
"optional": {
|
||||
"image1": (IO.IMAGE, {"tooltip": "参考图片1"}),
|
||||
"image2": (IO.IMAGE, {"tooltip": "参考图片2"}),
|
||||
"image3": (IO.IMAGE, {"tooltip": "参考图片3"}),
|
||||
"image4": (IO.IMAGE, {"tooltip": "参考图片4"}),
|
||||
"image5": (IO.IMAGE, {"tooltip": "参考图片5"}),
|
||||
"image6": (IO.IMAGE, {"tooltip": "参考图片6"}),
|
||||
"image7": (IO.IMAGE, {"tooltip": "参考图片7"}),
|
||||
"image_urls": (IO.STRING, {"default": "", "multiline": True, "tooltip": "参考图片URL列表,每行一个"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
|
||||
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
|
||||
|
||||
def generate(self, client, model, prompt, duration, aspect_ratio, resolution,
|
||||
image1=None, image2=None, image3=None, image4=None,
|
||||
image5=None, image6=None, image7=None,
|
||||
image_urls="", seed=0, bgm=False):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
# Collect images
|
||||
images = []
|
||||
for img in [image1, image2, image3, image4, image5, image6, image7]:
|
||||
if img is not None:
|
||||
images.append(image_to_base64(img))
|
||||
|
||||
# Add URL images
|
||||
if image_urls and image_urls.strip():
|
||||
for url in image_urls.strip().split('\n'):
|
||||
url = url.strip()
|
||||
if url:
|
||||
images.append(url)
|
||||
|
||||
if not images:
|
||||
raise ValueError("至少需要提供1张参考图片")
|
||||
if len(images) > 7:
|
||||
print(f"Warning: 最多支持7张参考图片,当前{len(images)}张,将只使用前7张")
|
||||
images = images[:7]
|
||||
|
||||
task_input = {
|
||||
"images": images,
|
||||
"prompt": prompt,
|
||||
}
|
||||
|
||||
parameters = {
|
||||
"vidu_type": "reference2video",
|
||||
"duration": duration,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
"resolution": resolution,
|
||||
"seed": seed,
|
||||
"bgm": bgm,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Vidu Ref2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Vidu_Reference2Video": ViduReference2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Vidu_Reference2Video": "Vidu Reference2Video",
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
"""
|
||||
Vidu StartEnd2Video - 首尾帧生视频模型
|
||||
Models: viduq3-pro, viduq3-turbo, viduq2-pro-fast, viduq2-pro, viduq2-turbo
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2-pro-fast", "viduq2-pro", "viduq2-turbo"]
|
||||
RESOLUTIONS = ["540p", "720p", "1080p"]
|
||||
MOVEMENT_AMPLITUDES = ["auto", "small", "medium", "large"]
|
||||
|
||||
|
||||
class ViduStartEnd2VideoNode:
|
||||
"""
|
||||
Vidu StartEnd2Video - 首尾帧生视频
|
||||
Models: viduq3-pro/viduq3-turbo (1-16s), viduq2-pro-fast/viduq2-pro/viduq2-turbo (1-8s)
|
||||
注意:首尾帧分辨率需相近 (0.8~1.25)
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro/turbo: 支持1-16秒; viduq2-pro-fast/pro/turbo: 支持1-8秒"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 1, "max": 16, "step": 1, "tooltip": "视频时长(秒),viduq3系列支持1-16秒,viduq2系列支持1-8秒"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
|
||||
"movement_amplitude": (MOVEMENT_AMPLITUDES, {"default": "auto", "tooltip": "运动幅度"}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
|
||||
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL"}),
|
||||
"last_frame_image": (IO.IMAGE, {"tooltip": "尾帧图片"}),
|
||||
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "尾帧图片URL"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "文本提示词,最长2000字符"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
|
||||
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
|
||||
|
||||
def generate(self, client, model, duration, resolution, movement_amplitude,
|
||||
first_frame_image=None, first_frame_url="",
|
||||
last_frame_image=None, last_frame_url="",
|
||||
prompt="", seed=0, bgm=False):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
# Validate first frame
|
||||
has_first_url = first_frame_url and first_frame_url.strip()
|
||||
has_first_image = first_frame_image is not None
|
||||
if has_first_url and has_first_image:
|
||||
raise ValueError("首帧:请提供 image 或 url,不能同时提供")
|
||||
if not has_first_url and not has_first_image:
|
||||
raise ValueError("必须提供首帧图片")
|
||||
|
||||
# Validate last frame
|
||||
has_last_url = last_frame_url and last_frame_url.strip()
|
||||
has_last_image = last_frame_image is not None
|
||||
if has_last_url and has_last_image:
|
||||
raise ValueError("尾帧:请提供 image 或 url,不能同时提供")
|
||||
if not has_last_url and not has_last_image:
|
||||
raise ValueError("必须提供尾帧图片")
|
||||
if duration > 8 and not model.startswith("viduq3-"):
|
||||
raise ValueError("只有 viduq3 系列模型支持超过8秒的视频时长")
|
||||
|
||||
task_input = {}
|
||||
|
||||
# First frame
|
||||
if has_first_url:
|
||||
task_input["first_frame_url"] = first_frame_url.strip()
|
||||
else:
|
||||
task_input["first_frame_url"] = image_to_base64(first_frame_image)
|
||||
|
||||
# Last frame
|
||||
if has_last_url:
|
||||
task_input["last_frame_url"] = last_frame_url.strip()
|
||||
else:
|
||||
task_input["last_frame_url"] = image_to_base64(last_frame_image)
|
||||
|
||||
if prompt and prompt.strip():
|
||||
task_input["prompt"] = prompt.strip()
|
||||
|
||||
parameters = {
|
||||
"vidu_type": "start-end2video",
|
||||
"duration": duration,
|
||||
"resolution": resolution,
|
||||
"movement_amplitude": movement_amplitude,
|
||||
"seed": seed,
|
||||
"bgm": bgm,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Vidu StartEnd2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Vidu_StartEnd2Video": ViduStartEnd2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Vidu_StartEnd2Video": "Vidu StartEnd2Video",
|
||||
}
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
"""
|
||||
Vidu Text2Video - 文生视频模型
|
||||
Models: viduq3-pro, viduq3-turbo, viduq2
|
||||
"""
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2"]
|
||||
ASPECT_RATIOS = ["16:9", "9:16", "3:4", "4:3", "1:1"]
|
||||
RESOLUTIONS = ["540p", "720p", "1080p"]
|
||||
|
||||
|
||||
class ViduText2VideoNode:
|
||||
"""
|
||||
Vidu Text2Video - 文生视频
|
||||
Models: viduq3-pro, viduq3-turbo, viduq2
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro: 效果好细节丰富, viduq3-turbo: 生成快, viduq2: 旧版模型"}),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "A beautiful sunset over the ocean", "tooltip": "文本提示词,最长2000字符"}),
|
||||
"duration": (IO.INT, {"default": 5, "min": 1, "max": 10, "step": 1, "tooltip": "视频时长(秒)"}),
|
||||
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "长宽比"}),
|
||||
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
|
||||
},
|
||||
"optional": {
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子,0表示随机"}),
|
||||
"guidance_scale": (IO.FLOAT, {"default": 7.5, "min": 1.0, "max": 20.0, "step": 0.5, "tooltip": "引导系数"}),
|
||||
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
|
||||
|
||||
def generate(self, client, model, prompt, duration, aspect_ratio, resolution, seed=0, guidance_scale=7.5, bgm=False):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
task_input = {"prompt": prompt}
|
||||
parameters = {
|
||||
"vidu_type": "text2video",
|
||||
"duration": duration,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
"resolution": resolution,
|
||||
"seed": seed,
|
||||
"guidance_scale": guidance_scale,
|
||||
"bgm": bgm,
|
||||
}
|
||||
|
||||
# Submit task
|
||||
submit_res = mv_client.submit_task(model, task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res}")
|
||||
|
||||
print(f"Vidu T2V task submitted: {task_id}")
|
||||
|
||||
# Poll for result
|
||||
video_url = self._poll_task(mv_client, task_id)
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
def _poll_task(self, mv_client, task_id, max_retries=180):
|
||||
for i in range(max_retries):
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise Exception("Task succeeded but no video URL returned")
|
||||
elif task_status == "Failure":
|
||||
error = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise Exception(f"Unknown status: {task_status}")
|
||||
|
||||
raise Exception("Task timed out")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Vidu_Text2Video": ViduText2VideoNode,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Vidu_Text2Video": "Vidu Text2Video",
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from .modelverse_api.utils import image_to_base64
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class Modelverse_WanAII2V:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "Convert to video","tooltip": "Text prompt to guide video generation"}),
|
||||
},
|
||||
"optional": {
|
||||
"first_frame_image": (IO.IMAGE,),
|
||||
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "First frame image URL (use either this OR first_frame_image, not both)"}),
|
||||
"last_frame_image": (IO.IMAGE,),
|
||||
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "Optional: URL for the last frame of the video"}),
|
||||
"negative_prompt": (IO.STRING, {"multiline": True, "default": "low quality, blurry","tooltip": "Negative prompt to avoid unwanted content"}),
|
||||
"resolution": (["720P", "480P"], {"default": "720P", "tooltip": "Output video resolution"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "Random seed for reproducible results"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate_video"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Wan"
|
||||
|
||||
def generate_video(self, client, prompt, first_frame_image=None, first_frame_url="", last_frame_image=None, last_frame_url="", negative_prompt="", resolution="720P", seed=0):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set in the client")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
# Prepare the input data
|
||||
task_input = {"prompt": prompt}
|
||||
|
||||
# Validate first frame input - must provide either image or URL, but not both
|
||||
has_url = first_frame_url and first_frame_url.strip()
|
||||
has_image = first_frame_image is not None
|
||||
|
||||
if has_url and has_image:
|
||||
raise ValueError("Please provide either first_frame_image OR first_frame_url, not both")
|
||||
elif not has_url and not has_image:
|
||||
raise ValueError("Must provide either first_frame_image or first_frame_url")
|
||||
|
||||
# Handle first frame
|
||||
if has_url:
|
||||
task_input["first_frame_url"] = first_frame_url.strip()
|
||||
print(f"Using first frame URL: {first_frame_url}")
|
||||
else:
|
||||
# Convert IMAGE tensor to base64
|
||||
first_frame_base64 = image_to_base64(first_frame_image)
|
||||
if not first_frame_base64:
|
||||
raise ValueError("Failed to convert first frame image to base64")
|
||||
task_input["first_frame_url"] = first_frame_base64
|
||||
print("Using first frame from IMAGE input (converted to base64)")
|
||||
|
||||
# Handle last frame (optional)
|
||||
if last_frame_url and last_frame_url.strip():
|
||||
task_input["last_frame_url"] = last_frame_url.strip()
|
||||
print(f"Using last frame URL: {last_frame_url}")
|
||||
elif last_frame_image is not None:
|
||||
# Convert IMAGE tensor to base64
|
||||
last_frame_base64 = image_to_base64(last_frame_image)
|
||||
if last_frame_base64:
|
||||
task_input["last_frame_url"] = last_frame_base64
|
||||
print("Using last frame from IMAGE input (converted to base64)")
|
||||
|
||||
# Add negative prompt if provided
|
||||
if negative_prompt and negative_prompt.strip():
|
||||
task_input["negative_prompt"] = negative_prompt.strip()
|
||||
|
||||
# Set parameters
|
||||
parameters = {
|
||||
"resolution": resolution,
|
||||
"duration": 5, # Fixed as per documentation
|
||||
"seed": seed,
|
||||
}
|
||||
|
||||
print(f"Submitting I2V task with model: Wan-AI/Wan2.2-I2V")
|
||||
print(f"Parameters: resolution={resolution}, seed={seed}")
|
||||
|
||||
# 1. Submit the task
|
||||
submit_res = mv_client.submit_task("Wan-AI/Wan2.2-I2V", task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res.get('request_id')}")
|
||||
|
||||
print(f"Task submitted successfully with ID: {task_id}")
|
||||
|
||||
# 2. Poll for the result
|
||||
video_url = ""
|
||||
max_retries = 120 # Maximum 10 minutes (120 * 5 seconds)
|
||||
retry_count = 0
|
||||
|
||||
while True:
|
||||
try:
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
urls = status_res.get("output", {}).get("urls", [])
|
||||
if urls and len(urls) > 0:
|
||||
video_url = urls[0]
|
||||
print(f"Task completed successfully! Video URL: {video_url}")
|
||||
break
|
||||
else:
|
||||
raise Exception("Task succeeded but no video URL was returned.")
|
||||
|
||||
elif task_status == "Failure":
|
||||
error_message = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error_message}")
|
||||
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id} is {task_status}, waiting... ({retry_count + 1}/{max_retries})")
|
||||
time.sleep(5) # Wait for 5 seconds before polling again
|
||||
retry_count += 1
|
||||
else:
|
||||
raise Exception(f"Unknown task status: {task_status}")
|
||||
|
||||
except Exception as e:
|
||||
if "Task failed" in str(e) or "Unknown task status" in str(e):
|
||||
raise e
|
||||
print(f"Error checking task status: {e}, retrying...")
|
||||
retry_count += 1
|
||||
time.sleep(5)
|
||||
|
||||
if not video_url:
|
||||
raise Exception(f"Task timed out after {max_retries * 5} seconds")
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse_WanAII2V": Modelverse_WanAII2V
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse_WanAII2V": "Modelverse Wan-AI I2V"
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
import time
|
||||
from .modelverse_api.client import ModelverseClient
|
||||
from comfy.comfy_types.node_typing import IO
|
||||
|
||||
|
||||
class Modelverse_WanAIT2V:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"client": ("MODELVERSE_API_CLIENT",),
|
||||
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower","tooltip": "Text prompt of the image to generate"}),
|
||||
"negative_prompt": (IO.STRING, {"multiline": True, "default": "low quality","tooltip": "Negative prompt of the image to generate"}),
|
||||
"resolution": (["720P", "480P"], {"default": "720P"}),
|
||||
"size": (["1280x720", "720x1280", "832x480", "480x832"], {"default": "1280x720"}),
|
||||
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = (IO.STRING, IO.STRING)
|
||||
RETURN_NAMES = ("url", "task_id")
|
||||
FUNCTION = "generate_video"
|
||||
CATEGORY = "UCLOUD_MODELVERSE/Wan"
|
||||
|
||||
def generate_video(self, client, prompt, negative_prompt, resolution, size, seed):
|
||||
api_key = client.get("api_key")
|
||||
if not api_key:
|
||||
raise ValueError("API key is not set in the client")
|
||||
|
||||
mv_client = ModelverseClient(api_key)
|
||||
|
||||
task_input = {"prompt": prompt}
|
||||
if negative_prompt:
|
||||
task_input["negative_prompt"] = negative_prompt
|
||||
|
||||
parameters = {
|
||||
"resolution": resolution,
|
||||
"size": size,
|
||||
"seed": seed,
|
||||
"duration": 5 # Fixed as per documentation
|
||||
}
|
||||
|
||||
# 1. Submit the task
|
||||
submit_res = mv_client.submit_task("Wan-AI/Wan2.2-T2V", task_input, parameters)
|
||||
task_id = submit_res.get("output", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise Exception(f"Failed to submit task: {submit_res.get('request_id')}")
|
||||
|
||||
# 2. Poll for the result
|
||||
video_url = ""
|
||||
while True:
|
||||
status_res = mv_client.get_task_status(task_id)
|
||||
task_status = status_res.get("output", {}).get("task_status")
|
||||
|
||||
if task_status == "Success":
|
||||
video_url = status_res.get("output", {}).get("urls", [None])[0]
|
||||
if not video_url:
|
||||
raise Exception("Task succeeded but no video URL was returned.")
|
||||
break
|
||||
elif task_status == "Failure":
|
||||
error_message = status_res.get("output", {}).get("error_message", "Unknown error")
|
||||
raise Exception(f"Task failed: {error_message}")
|
||||
elif task_status in ["Pending", "Running"]:
|
||||
print(f"Task {task_id} is {task_status}, waiting...")
|
||||
time.sleep(5) # Wait for 5 seconds before polling again
|
||||
else:
|
||||
raise Exception(f"Unknown task status: {task_status}")
|
||||
|
||||
return (video_url, task_id)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Modelverse_WanAIT2V": Modelverse_WanAIT2V
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Modelverse_WanAIT2V": "Modelverse Wan-AI T2V"
|
||||
}
|
||||
|
||||
+3
-3
@@ -1,9 +1,9 @@
|
||||
[project]
|
||||
name = "ucloud-comfyui"
|
||||
description = "This is a set of ComfyUI nodes for UCloud Moderlverse Image generation model API."
|
||||
version = "1.0.2"
|
||||
description = "This is a custom node for ComfyUI that allows you to use the UCloud API directly in ComfyUI. UCloud is a high-performance AI computing platform providing powerful image generation models and industry-leading inference speeds. It offers a wide range of AI model options, including popular text-to-image and image editing models, ready to use without local GPU setup."
|
||||
version = "1.0.9"
|
||||
license = {file = "LICENSE"}
|
||||
dependencies = ["requests", "pillow"]
|
||||
dependencies = ["requests", "pillow", "numpy", "openai"]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/ucloud/comfyui-plugin"
|
||||
|
||||
@@ -1,2 +1,5 @@
|
||||
requests >= 2.32.4
|
||||
pillow >= 11
|
||||
numpy >= 2.0
|
||||
openai >= 1.107.0
|
||||
GitPython==3.1.45
|
||||
@@ -0,0 +1,136 @@
|
||||
.mv-secret-overlay {
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: 9999;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: rgba(0, 0, 0, 0.58);
|
||||
}
|
||||
|
||||
.mv-secret-dialog {
|
||||
width: min(760px, 92vw);
|
||||
max-height: 82vh;
|
||||
padding: 18px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
color: #f0f0f0;
|
||||
background: #171717;
|
||||
border: 1px solid #3f3f3f;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 18px 60px rgba(0, 0, 0, 0.45);
|
||||
font-family: sans-serif;
|
||||
}
|
||||
|
||||
.mv-secret-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.mv-secret-title {
|
||||
margin: 0;
|
||||
font-size: 16px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.mv-secret-close {
|
||||
border: 0;
|
||||
color: #bdbdbd;
|
||||
background: transparent;
|
||||
font-size: 18px;
|
||||
line-height: 1;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.mv-secret-rows {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.mv-secret-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(130px, 0.7fr) minmax(180px, 1.3fr) auto auto;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.mv-secret-input {
|
||||
min-width: 0;
|
||||
height: 30px;
|
||||
padding: 5px 8px;
|
||||
border: 1px solid #555;
|
||||
border-radius: 4px;
|
||||
color: #eee;
|
||||
background: #262626;
|
||||
font-size: 13px;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.mv-secret-input:focus {
|
||||
border-color: #7d8cff;
|
||||
}
|
||||
|
||||
.mv-secret-button,
|
||||
.mv-secret-add {
|
||||
height: 30px;
|
||||
padding: 5px 9px;
|
||||
border-radius: 4px;
|
||||
color: #eee;
|
||||
font-size: 13px;
|
||||
cursor: pointer;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.mv-secret-save {
|
||||
background: #2f6336;
|
||||
border: 1px solid #4c9b55;
|
||||
}
|
||||
|
||||
.mv-secret-delete {
|
||||
background: #653131;
|
||||
border: 1px solid #a44;
|
||||
}
|
||||
|
||||
.mv-secret-add {
|
||||
width: 100%;
|
||||
text-align: left;
|
||||
color: #ccc;
|
||||
background: #222;
|
||||
border: 1px dashed #5a5a5a;
|
||||
}
|
||||
|
||||
.mv-secret-save-notice {
|
||||
margin-top: 4px;
|
||||
min-height: 18px;
|
||||
color: #ff4d4f;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
text-align: center;
|
||||
opacity: 0;
|
||||
transition: opacity 0.4s ease;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.mv-secret-save-notice-visible {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.mv-secret-save-notice-fade {
|
||||
opacity: 0;
|
||||
}
|
||||
|
||||
@media (max-width: 640px) {
|
||||
.mv-secret-row {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.mv-secret-button,
|
||||
.mv-secret-add {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { api } from "../../scripts/api.js";
|
||||
|
||||
const cssUrl = new URL("./modelverse_secrets.css", import.meta.url);
|
||||
if (!document.querySelector(`link[href="${cssUrl}"]`)) {
|
||||
document.head.appendChild(Object.assign(document.createElement("link"), { rel: "stylesheet", href: cssUrl }));
|
||||
}
|
||||
|
||||
const NODE_NAME = "UCloud ModelVerse Secret Client";
|
||||
const WIDGET_NAME = "secret";
|
||||
const ENDPOINT = "/modelverse-secrets";
|
||||
|
||||
let cachedSecretNames = [];
|
||||
|
||||
function el(tag, props = {}, ...children) {
|
||||
const node = Object.assign(document.createElement(tag), props);
|
||||
node.append(...children);
|
||||
return node;
|
||||
}
|
||||
|
||||
async function fetchSecrets() {
|
||||
const response = await api.fetchApi(ENDPOINT, { method: "GET" });
|
||||
return await response.json();
|
||||
}
|
||||
|
||||
async function refreshSecretNamesCache() {
|
||||
const secrets = await fetchSecrets();
|
||||
cachedSecretNames = Object.keys(secrets);
|
||||
return cachedSecretNames;
|
||||
}
|
||||
|
||||
async function refreshSecretDropdowns(preferredValue) {
|
||||
const names = await refreshSecretNamesCache();
|
||||
for (const node of app.graph._nodes ?? []) {
|
||||
if (node.comfyClass !== NODE_NAME && node.type !== NODE_NAME) {
|
||||
continue;
|
||||
}
|
||||
const widget = node.widgets?.find((item) => item.name === WIDGET_NAME);
|
||||
if (!widget) {
|
||||
continue;
|
||||
}
|
||||
widget.options.values = names.slice();
|
||||
if (preferredValue && names.includes(preferredValue)) {
|
||||
widget.value = preferredValue;
|
||||
} else if (names.length && !names.includes(widget.value)) {
|
||||
widget.value = names[0];
|
||||
} else if (!names.length) {
|
||||
widget.value = "";
|
||||
}
|
||||
if (typeof widget.callback === "function") {
|
||||
try { widget.callback(widget.value); } catch (e) { /* noop */ }
|
||||
}
|
||||
node.setDirtyCanvas?.(true, true);
|
||||
}
|
||||
app.graph?.setDirtyCanvas?.(true, true);
|
||||
}
|
||||
|
||||
function buildSecretRow(name = "", value = "", isNew = false, onSaved) {
|
||||
let originalName = name;
|
||||
const nameInput = el("input", { className: "mv-secret-input", placeholder: "Secret name", value: name });
|
||||
const valueInput = el("input", { className: "mv-secret-input", placeholder: "API key", value, type: "password", autocomplete: "off" });
|
||||
|
||||
const saveButton = el("button", { className: "mv-secret-button mv-secret-save", textContent: isNew ? "Add" : "Save" });
|
||||
const deleteButton = el("button", { className: "mv-secret-button mv-secret-delete", textContent: "Delete" });
|
||||
const row = el("div", { className: "mv-secret-row" }, nameInput, valueInput, saveButton, deleteButton);
|
||||
|
||||
saveButton.onclick = async () => {
|
||||
const key = nameInput.value.trim();
|
||||
if (!key) {
|
||||
nameInput.focus();
|
||||
return;
|
||||
}
|
||||
|
||||
const response = await api.fetchApi(ENDPOINT, {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ key, value: valueInput.value }),
|
||||
});
|
||||
if (!response.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!isNew && originalName && originalName !== key) {
|
||||
await api.fetchApi(`${ENDPOINT}/${encodeURIComponent(originalName)}`, { method: "DELETE" });
|
||||
}
|
||||
|
||||
originalName = key;
|
||||
await refreshSecretDropdowns(key);
|
||||
|
||||
if (typeof onSaved === "function") {
|
||||
onSaved();
|
||||
}
|
||||
|
||||
if (isNew) {
|
||||
row.replaceWith(buildSecretRow(key, valueInput.value, false, onSaved));
|
||||
return;
|
||||
}
|
||||
|
||||
saveButton.textContent = "Saved";
|
||||
setTimeout(() => {
|
||||
saveButton.textContent = "Save";
|
||||
}, 1200);
|
||||
};
|
||||
|
||||
deleteButton.onclick = async () => {
|
||||
if (!originalName) {
|
||||
row.remove();
|
||||
return;
|
||||
}
|
||||
const response = await api.fetchApi(`${ENDPOINT}/${encodeURIComponent(originalName)}`, { method: "DELETE" });
|
||||
if (response.ok) {
|
||||
row.remove();
|
||||
await refreshSecretDropdowns();
|
||||
}
|
||||
};
|
||||
|
||||
return row;
|
||||
}
|
||||
|
||||
function createSecretsModal(secrets) {
|
||||
const saveNotice = el("div", { className: "mv-secret-save-notice" });
|
||||
const showSaveNotice = () => {
|
||||
saveNotice.textContent = "Once added, there is a 10-second delay before the item shows up in the dropdown.";
|
||||
saveNotice.classList.remove("mv-secret-save-notice-fade");
|
||||
void saveNotice.offsetWidth;
|
||||
saveNotice.classList.add("mv-secret-save-notice-visible");
|
||||
clearTimeout(saveNotice._hideTimer);
|
||||
saveNotice._hideTimer = setTimeout(() => {
|
||||
saveNotice.classList.add("mv-secret-save-notice-fade");
|
||||
}, 10000);
|
||||
};
|
||||
const rows = el(
|
||||
"div",
|
||||
{ className: "mv-secret-rows" },
|
||||
...Object.entries(secrets).map(([name, value]) => buildSecretRow(name, value, false, showSaveNotice)),
|
||||
);
|
||||
const addButton = el("button", {
|
||||
className: "mv-secret-add",
|
||||
textContent: "+ Add Secret",
|
||||
onclick: () => {
|
||||
const row = buildSecretRow("", "", true, showSaveNotice);
|
||||
rows.appendChild(row);
|
||||
row.querySelector("input")?.focus();
|
||||
},
|
||||
});
|
||||
const closeButton = el("button", { className: "mv-secret-close", textContent: "x" });
|
||||
|
||||
const overlay = el("div", { className: "mv-secret-overlay" },
|
||||
el("div", { className: "mv-secret-dialog" },
|
||||
el("div", { className: "mv-secret-header" },
|
||||
el("h3", { className: "mv-secret-title", textContent: "Modelverse Secrets" }),
|
||||
closeButton,
|
||||
),
|
||||
rows,
|
||||
addButton,
|
||||
saveNotice,
|
||||
),
|
||||
);
|
||||
|
||||
closeButton.onclick = () => overlay.remove();
|
||||
overlay.addEventListener("click", (event) => {
|
||||
if (event.target === overlay) {
|
||||
overlay.remove();
|
||||
}
|
||||
});
|
||||
|
||||
return overlay;
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "compshare.modelverse_secrets",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== NODE_NAME && node.type !== NODE_NAME) {
|
||||
return;
|
||||
}
|
||||
|
||||
const names = await refreshSecretNamesCache();
|
||||
const index = node.widgets?.findIndex((item) => item.name === WIDGET_NAME) ?? -1;
|
||||
const savedValue = index >= 0 ? node.widgets[index].value : undefined;
|
||||
if (index >= 0) {
|
||||
node.widgets.splice(index, 1);
|
||||
}
|
||||
|
||||
const defaultValue = savedValue && names.includes(savedValue) ? savedValue : (names[0] ?? "");
|
||||
const comboOptions = {
|
||||
get values() {
|
||||
return cachedSecretNames.slice();
|
||||
},
|
||||
set values(v) { /* ignore external writes; cache is the source of truth */ },
|
||||
};
|
||||
node.addWidget("combo", WIDGET_NAME, defaultValue, () => {}, comboOptions);
|
||||
node.addWidget("button", "Edit Secrets", null, async () => {
|
||||
await refreshSecretNamesCache();
|
||||
document.body.appendChild(createSecretsModal(await fetchSecrets()));
|
||||
});
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user