21 Commits
Author SHA1 Message Date
Jianfeng Mei 54a504e122 Merge pull request #7 from yinjiajiu/main
feature:添加Kling,seedance,veo,happyhorse,vidu和Gemini系列模型,增加api key配置,防止泄漏
2026-06-11 14:46:56 +08:00
yinjiajiu 36876b888f feature:添加Kling,seedance,veo,happyhorse,vidu和Gemini系列模型,增加api key配置,防止泄漏 2026-06-11 14:43:02 +08:00
jason af42216bd0 Merge pull request #6 from mfzzf/main
feat(modelverse): add Gemini Pro Image and Vidu & Sora video generation support
2026-01-04 13:59:18 +08:00
jason.mei 89f0ef3fe6 chore: update version to 1.0.8 in pyproject.toml 2026-01-04 13:55:14 +08:00
jason.mei db0b8fa266 feat(sora): add Sora2 text-to-video and image-to-video generation nodes
- Add SoraText2VideoNode for OpenAI Sora2 text-to-video generation
* Support both standard and Pro model variants
* Configurable video size, duration, and prompt parameters
* Pro version supports extended resolution options (1024x1792, 1792x1024)
- Add SoraImg2VideoNode for OpenAI Sora2 image-to-video generation
* Support both standard and Pro model variants
* Accept first frame as image or URL input
* Pro version supports 1080p resolution
- Implement async task polling with configurable retry logic
- Integrate with ModelverseClient for API communication
- Add comprehensive input validation and error handling
- Follow established video generation patterns from Vidu and Wan AI nodes
2025-12-31 10:02:52 +08:00
jason.mei 4092be6a0a refactor(preview_video): simplify video saving and reorder download logic
- Add requests import for direct HTTP video downloads
- Replace complex video object data extraction with direct URL download using requests.get()
- Move file saving logic before download_url_to_video_output call to avoid consuming data stream
- Simplify video data retrieval by downloading directly from URL instead of extracting from video object
- Add timeout parameter (120s) and error handling for HTTP requests
- Improve code clarity by removing conditional checks for different video object attributes
2025-12-30 11:02:47 +08:00
jason.mei 1f493b180e refactor(preview_video): remove unused API client imports
- Remove unused ApiEndpoint, HttpMethod, and SynchronousOperation imports
- Simplify module dependencies by eliminating unnecessary API client references
- Maintain existing functionality with cleaner import structure
2025-12-30 10:49:51 +08:00
jason.mei fafef977a1 refactor(preview_video): migrate to async API and use comfy video types
- Replace requests library with comfy_api_nodes download utility for better integration
- Convert run method to async for non-blocking video download operations
- Update input/output types to use IO enum (IO.STRING, IO.BOOLEAN, IO.VIDEO)
- Change return type from STRING file path to VIDEO object for direct video output
- Simplify video handling by using download_url_to_video_output utility function
- Add support for multiple video data access patterns (video_io, get_bytes, BytesIO)
- Improve video saving logic with proper data extraction from video object
- Remove unused os.path and re imports optimization for cleaner code
- Maintains backward compatibility with save_output parameter for optional file persistence
2025-12-30 10:47:38 +08:00
jason.mei 8715c032ae feat(vidu,wan_ai): refactor video generation to return URLs and add preview node
- Add new ModelversePreviewVideo node for downloading and previewing generated videos
- Change all Vidu video generation nodes (extend, i2v, ref2v, startend2v, t2v) to return video URL and task ID instead of downloaded video objects
- Change all Wan AI video generation nodes (i2v, t2v) to return video URL and task ID instead of downloaded video objects
- Remove duplicate VideoFromFile import fallback logic from all video generation modules
- Simplify video handling by delegating download responsibility to preview node
- Improve separation of concerns between generation and preview workflows
2025-12-30 10:29:46 +08:00
jason.mei 7da714c686 refactor(vidu,wan_ai): add fallback VideoFromFile wrapper for import compatibility
- Add third-level fallback import handler for VideoFromFile class across all video modules
- Implement simple VideoFromFile wrapper class with default dimensions (1280x720) when imports fail
- Support video_io objects with getvalue() method for BytesIO compatibility
- Add clarifying comments for each import path (newer, older, and fallback)
- Apply consistent fallback pattern to vidu_extend, vidu_i2v, vidu_ref2v, vidu_startend2v, vidu_t2v, wan_ai_i2v, and wan_ai_t2v modules
- Improve robustness for different ComfyUI versions and installation configurations
2025-12-29 18:36:16 +08:00
jason.mei ee2debe867 refactor(vidu,wan_ai): remove duplicate VideoFromFile fallback implementation
- Remove redundant VideoFromFile class implementations from vidu_extend.py, vidu_i2v.py, vidu_ref2v.py, vidu_startend2v.py, and vidu_t2v.py
- Remove duplicate VideoFromFile fallback from wan_ai_i2v.py and wan_ai_t2v.py
- Simplify import logic by relying on comfy library's VideoFromFile implementation
- Reduce code duplication across multiple video generation modules
- Improve maintainability by centralizing video handling logic
2025-12-29 18:32:21 +08:00
jason.mei c2d2ec96b2 feat(vidu,wan_ai): add video dimension extraction with av library
- Add `get_dimensions()` method to VideoFromFile fallback class across all vidu and wan_ai modules
- Implement PyAV (av) library integration to extract video width and height from container streams
- Add dimension caching with `_width` and `_height` attributes to avoid redundant parsing
- Implement fallback dimensions (1280x720) when video parsing fails
- Add proper stream seeking to reset file position after dimension extraction
- Update requirements.txt to include av library dependency
- Ensure consistent video dimension handling across vidu_t2v, vidu_i2v, vidu_ref2v, vidu_startend2v, vidu_extend, wan_ai_t2v, and wan_ai_i2v modules
2025-12-29 18:11:04 +08:00
jason.mei 20093d1c40 feat(wan_ai_i2v): simplify video handling and improve error management
- Remove unused imports (os, tempfile) to reduce dependencies
- Simplify VideoFromFile import fallback logic by removing unnecessary comments
- Remove get_dimensions() method from VideoFromFile wrapper class
- Consolidate video download logic and remove redundant error handling
- Increase download timeout from 60s to 120s for better reliability
- Remove verbose print statements for cleaner logging
- Simplify video data processing by directly using response.content
- Streamline return statement to directly instantiate VideoFromFile
- Remove fallback video_data return path for consistent API behavior
- Improve code readability and maintainability through simplification
2025-12-29 17:54:47 +08:00
jason.mei 7a3091df10 feat(wan_ai_t2v): improve video generation and error handling
- Remove unused imports (os, tempfile)
- Add fallback VideoFromFile class for better compatibility
- Simplify video download and processing logic
- Add timeout parameter (120s) to video download request
- Improve error handling with specific ImportError catching
- Reduce code complexity by removing redundant checks and logging
- Streamline BytesIO object creation and return statement
- Enhance robustness when VideoFromFile is not available in ComfyUI
2025-12-29 17:53:31 +08:00
jason.mei bd7d00b1ce feat(modelverse): add Gemini Pro Image and Vidu video generation support
- Add Gemini 3 Pro Image Preview (Nano Banana Pro) ComfyUI node with high-resolution output (1K, 2K, 4K)
- Add GeminiProImageRequest builder supporting aspect ratio control, Google Search grounding, and up to 14 reference images
- Add Vidu video generation nodes: t2v, i2v, ref2v, and startend2v for comprehensive video synthesis
- Add Vidu extend node for video continuation and frame extension
- Update .gitignore to exclude .ace-tool/ and docs/ directories
- Enable professional asset production with advanced image and video generation capabilities
2025-12-29 16:03:39 +08:00
jasonandjason.mei 2be7d1e881 add gpt-image,qwen,nano banana (#5)
* add gitignore

* add video api

* fix: export to video

* fix

* fix

* fix

* fix

* fix

* add i2v

* fix

* fix

* add openai

* fix: add base url

* change default models

* fix: add base url

* add v1

* fix models

* fix models

* fix models

* fix models

* fix models

* fix bug

* fix

* try to fix

* use IO.STRING test

* image fix

* add chat content

* add files

* fix optional

* fix file read

* fix file read

* change name

* add get models

* add unused apikey

* change model sort

* fix IMAGE_LIST

* fix: change to image

* add model cache

* remove json_object

* change max_tokens to 65536

* change max_tokens to 6000

* add pic model

* fix gpt-image-1

* fix gpt-image-1

* fix gpt

* fix gpt

* fix gpt

* fix gpt

* add gpt edit

* fix gpt edit

* update project.toml

---------

Co-authored-by: jason.mei <jason.mei@ucloud.cn>
2025-09-23 11:26:19 +08:00
宋博瀚 568f2b2e15 Update pyproject.toml 2025-09-15 15:31:56 +08:00
jasonandjason.mei 681f6ebc5f support text generation
* add gitignore

* add video api

* fix: export to video

* fix

* fix

* fix

* fix

* fix

* add i2v

* fix

* fix

* add openai

* fix: add base url

* change default models

* fix: add base url

* add v1

* fix models

* fix models

* fix models

* fix models

* fix models

* fix bug

* fix

* try to fix

* use IO.STRING test

* image fix

* add chat content

* add files

* fix optional

* fix file read

* fix file read

* change name

* add get models

* add unused apikey

* change model sort

* fix IMAGE_LIST

* fix: change to image

* add model cache

* remove json_object

* change max_tokens to 65536

* change max_tokens to 6000

---------

Co-authored-by: jason.mei <jason.mei@ucloud.cn>
2025-09-15 15:31:26 +08:00
宋博瀚 3492952986 upgrade version 2025-09-09 13:53:21 +08:00
jasonandjason.mei 7d350330e7 add video async api
* add gitignore

* add video api

* fix: export to video

* fix

* fix

* fix

* fix

* fix

* add i2v

* fix

* fix

---------

Co-authored-by: jason.mei <jason.mei@ucloud.cn>
2025-09-09 13:52:07 +08:00
宋博瀚 cde31dc9a7 comfyui ucloud描述修改 2025-08-14 16:48:46 +08:00
53 changed files with 5278 additions and 128 deletions
+5
View File
@@ -3,3 +3,8 @@ __pycache__
py/__pycache__
py/modelverse_api/__pycache__
py/modelverse_api/requests/__pycache__
/venv
.idea/
.ace-tool/
docs
secrets.json
+20 -4
View File
@@ -15,13 +15,13 @@ This plugin provides seamless integration with UCloud's ModelVerse API, enabling
2. **Configure API Key**
- Get your API key: [Console En](https://console.ucloud-global.com/modelverse/experience/api-keys), [Console Zh](https://console.ucloud.cn/modelverse/experience/api-keys)
- Add your UCloud ModelVerse API key in the `Modelverse Client` node
- Alternatively, create a `config.ini` file with `MODELVERSE_API_KEY=your_api_key`
- Get your API key: [Console En](https://astraflow.ucloud.cn/modelverse/api-keys), [Console Zh](https://astraflow.ucloud.cn/modelverse/api-keys)
- Add your UCloud ModelVerse API key in the `Modelverse Secret Client` node
- Or Alternatively, create a `config.ini` file with `MODELVERSE_API_KEY=your_api_key`
### Basic Usage
1. Add a `Modelverse Client` node to your workflow
1. Add a `Modelverse Secret Client` node to your workflow
2. Choose your desired model from the supported models below
3. Connect the appropriate input/output nodes
4. Run your workflow
@@ -30,6 +30,8 @@ This plugin provides seamless integration with UCloud's ModelVerse API, enabling
Our plugin supports the following models, each optimized for specific use cases:
### Image Generation
- **Flux Dev** - Foundation model with strong stylization capabilities
- **Flux Kontext Pro Text2Image** - Specialized for text-to-image generation
- **Flux Kontext Pro** - Expert single image editing and enhancement
@@ -38,6 +40,20 @@ Our plugin supports the following models, each optimized for specific use cases:
- **Flux Kontext Max** - Flagship single image editing with advanced features
- **Flux Kontext Max (Multi-inputs)** - Most powerful multi-image processing
- **Step1X-Edit** - Precision local editing and inpainting
- **GPT Image 1** - OpenAI's text-to-image generation
- **Gemini 3 Pro Image** - Google's professional asset production with search grounding
- **Qwen Image** - Alibaba's text-to-image generation
### Video Generation
- **Kling V3** - Unified text/image-to-video and motion control
- **Kling V3 Omni** - Multimodal video generation with reference inputs
- **Doubao Seedance 2.0** - Text/image-to-video with audio generation
- **OpenAI Sora 2** - High-quality text-to-video generation
- **HappyHorse 1.0** - Text-to-video generation
- **Veo 3.1** - Google's video generation (text-to-video, image-to-video)
- **Vidu** - Multiple models for text/image-to-video (viduq3-pro, viduq3-turbo, viduq2)
- **Wan-AI** - Text/image-to-video generation
Note: (Multi-inputs) models use the same node interface as their single-input counterparts. Check the example workflows below for implementation details.
+2 -1
View File
@@ -39,7 +39,8 @@ for file in files:
serialized_CLASS_MAPPINGS = {k: serialize(v) for k, v in imported_module.NODE_CLASS_MAPPINGS.items()}
serialized_DISPLAY_NAME_MAPPINGS = {k: serialize(v) for k, v in imported_module.NODE_DISPLAY_NAME_MAPPINGS.items()}
all_nodes[file]={"NODE_CLASS_MAPPINGS": serialized_CLASS_MAPPINGS, "NODE_DISPLAY_NAME_MAPPINGS": serialized_DISPLAY_NAME_MAPPINGS}
except:
except Exception as e:
print(f"Failed to import {file}: {e}")
pass
+141
View File
@@ -0,0 +1,141 @@
"""
Doubao Seedance 2.0 - Text/image-to-video model
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.doubao_seedance_2 import (
DoubaoSeedance2,
MODEL,
RESOLUTIONS,
RATIOS,
)
from comfy.comfy_types.node_typing import IO
class DoubaoSeedance2Node:
"""
Doubao Seedance 2.0 text/image-to-video generation.
Supports text, first/last frame images, reference image/video/audio inputs.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video (recommended: up to 500 characters)",
}),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image"}),
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "First frame image URL (use either this OR first_frame_image, not both)"}),
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image"}),
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "Last frame image URL (use either this OR last_frame_image, not both)"}),
"reference_image": (IO.IMAGE, {"tooltip": "Reference image"}),
"reference_image_url": (IO.STRING, {"default": "", "tooltip": "Reference image URL (use either this OR reference_image, not both)"}),
"reference_video_url": (IO.STRING, {"default": "", "tooltip": "Reference video URL"}),
"reference_audio_url": (IO.STRING, {"default": "", "tooltip": "Reference audio URL"}),
"duration": (IO.INT, {"default": 5, "min": 4, "max": 15, "step": 1, "tooltip": "Video duration in seconds (4–15)"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "Output resolution; draft mode supports 480p only"}),
"ratio": (RATIOS, {"default": "adaptive", "tooltip": "Aspect ratio; adaptive auto-selects the best fit"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "Random seed for reproducible results"}),
"generate_audio": (IO.BOOLEAN, {"default": False, "tooltip": "Generate audio synchronized with the video"}),
"camera_fixed": (IO.BOOLEAN, {"default": False, "tooltip": "Fix camera position (no camera movement)"}),
"watermark": (IO.BOOLEAN, {"default": False, "tooltip": "Add watermark to the output video"}),
"draft": (IO.BOOLEAN, {"default": False, "tooltip": "Draft/preview mode (480p only)"}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Seedance"
def generate(
self,
client,
prompt,
first_frame_image=None,
first_frame_url="",
last_frame_image=None,
last_frame_url="",
reference_image=None,
reference_image_url="",
reference_video_url="",
reference_audio_url="",
duration=5,
resolution="720p",
ratio="adaptive",
seed=0,
generate_audio=False,
camera_fixed=False,
watermark=False,
draft=False,
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
request = DoubaoSeedance2(
prompt=prompt,
first_frame=first_frame_image,
first_frame_url=first_frame_url,
last_frame=last_frame_image,
last_frame_url=last_frame_url,
reference_image=reference_image,
reference_image_url=reference_image_url,
reference_video_url=reference_video_url,
reference_audio_url=reference_audio_url,
duration=duration,
resolution=resolution,
ratio=ratio,
seed=seed,
generate_audio=generate_audio,
camera_fixed=camera_fixed,
watermark=watermark,
draft=draft,
)
mv_client = ModelverseClient(api_key)
print(f"Submitting Seedance 2.0 task: model={MODEL}, prompt={prompt!r}")
submit_res = mv_client.submit_task_request(request)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Seedance 2.0 task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Doubao_Seedance_2": DoubaoSeedance2Node,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Doubao_Seedance_2": "Modelverse Doubao Seedance 2.0",
}
+12 -12
View File
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.flux_dev import FluxDev
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class FluxDevNode:
@@ -17,8 +17,8 @@ class FluxDevNode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"width": ("INT", {
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"width": (IO.INT, {
"default": 1024,
"min": 512,
"max": 1536,
@@ -26,7 +26,7 @@ class FluxDevNode:
"display": "number",
"tooltip": "Image width (512 to 1536)"
}),
"height": ("INT", {
"height": (IO.INT, {
"default": 1024,
"min": 512,
"max": 1536,
@@ -34,7 +34,7 @@ class FluxDevNode:
"display": "number",
"tooltip": "Image height (512 to 1536)"
}),
"strength": ("FLOAT", {
"strength": (IO.FLOAT, {
"default": 0.8,
"min": 0.01,
"max": 1.0,
@@ -42,14 +42,14 @@ class FluxDevNode:
"display": "number",
"tooltip": "Strength of the image-to-image transformation (0.01 to 1.0)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"num_images": ("INT", {
"num_images": (IO.INT, {
"default": 1,
"min": 1,
"max": 4,
@@ -57,7 +57,7 @@ class FluxDevNode:
"display": "number",
"tooltip": "Number of images to generate in a single request (1 to 4)"
}),
"num_requests": ("INT", {
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -65,7 +65,7 @@ class FluxDevNode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"num_inference_steps": ("INT", {
"num_inference_steps": (IO.INT, {
"default": 28,
"min": 1,
"max": 50,
@@ -73,7 +73,7 @@ class FluxDevNode:
"display": "number",
"tooltip": "Number of inference steps (1 to 50)"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 3.5,
"min": 0.0,
"max": 10.0,
@@ -83,7 +83,7 @@ class FluxDevNode:
}),
},
"optional": {
"image": ("IMAGE", {
"image": (IO.IMAGE, {
"tooltip": "The image for reference.",
"forceInput": False,
"default": None
@@ -94,7 +94,7 @@ class FluxDevNode:
RETURN_TYPES = ("IMAGE", )
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE"
CATEGORY = "UCLOUD_MODELVERSE/Flux"
FUNCTION = "execute"
async def execute(self,
+16 -9
View File
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.flux_kontext_max import FluxKontextMax, FluxKontextMaxMulti
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class FluxKontextMaxNode:
@@ -21,13 +21,13 @@ class FluxKontextMaxNode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"images": ("IMAGE,IMAGE_LIST", {
"tooltip": "The image(s) to edit from. If not included, use text-to-image mode.",
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"images": (IO.IMAGE, {
"tooltip": "Image(s) to edit. Connect a single IMAGE, or use Modelverse Image Packer for multiple images.",
"forceInput": False,
"default": None
}),
"num_requests": ("INT", {
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -35,14 +35,14 @@ class FluxKontextMaxNode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 2.5,
"min": 0.0,
"max": 10.0,
@@ -74,9 +74,16 @@ class FluxKontextMaxNode:
mode = "single"
if isinstance(images, list):
if len(images) > 1:
print("INFO:", "Running Flux Kontext Max multi-image edit mode.")
print("INFO:", f"{len(images)} image included for the multi-image edit.")
mode = "multi"
else:
images = images[0]
elif isinstance(images, torch.Tensor) and images.ndim == 4 and images.shape[0] > 1:
images = [images[i:i + 1] for i in range(images.shape[0])]
print("INFO:", "Running Flux Kontext Max multi-image edit mode.")
print(
"INFO:", f"{len(images)} image included for the multi-image edit.")
print("INFO:", f"{len(images)} image included for the multi-image edit.")
mode = "multi"
else:
print("INFO:", "Running Flux Kontext Max single-image edit mode.")
+7 -8
View File
@@ -2,8 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.flux_kontext_max import FluxKontextMaxT2I
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class FluxKontextMaxT2INode:
"""
@@ -18,12 +17,12 @@ class FluxKontextMaxT2INode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"aspect_ratio": (["21:9", "16:9", "16:10", "4:3", "1:1", "3:4", "10:16", "9:16", "9:21"], {
"default": "1:1",
"tooltip": "The aspect ratio of the output image, ranging from \"21:9\" to \"9:21\", default is \"1:1\""
}),
"num_images": ("INT", {
"num_images": (IO.INT, {
"default": 1,
"min": 1,
"max": 4,
@@ -31,7 +30,7 @@ class FluxKontextMaxT2INode:
"display": "number",
"tooltip": "Number of images to generate in a single request (1 to 4)"
}),
"num_requests": ("INT", {
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -39,14 +38,14 @@ class FluxKontextMaxT2INode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 2.5,
"min": 0.0,
"max": 10.0,
@@ -60,7 +59,7 @@ class FluxKontextMaxT2INode:
RETURN_TYPES = ("IMAGE", )
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE"
CATEGORY = "UCLOUD_MODELVERSE/Flux"
FUNCTION = "execute"
async def execute(self,
+17 -10
View File
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.flux_kontext_pro import FluxKontextPro, FluxKontextProMulti
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class FluxKontextProNode:
@@ -21,13 +21,13 @@ class FluxKontextProNode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"images": ("IMAGE,IMAGE_LIST", {
"tooltip": "The image(s) to edit from. If not included, use text-to-image mode.",
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"images": (IO.IMAGE, {
"tooltip": "Image(s) to edit. Connect a single IMAGE, or use Modelverse Image Packer for multiple images.",
"forceInput": False,
"default": None
}),
"num_requests": ("INT", {
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -35,14 +35,14 @@ class FluxKontextProNode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 2.5,
"min": 0.0,
"max": 10.0,
@@ -56,7 +56,7 @@ class FluxKontextProNode:
RETURN_TYPES = ("IMAGE", )
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE"
CATEGORY = "UCLOUD_MODELVERSE/Flux"
FUNCTION = "execute"
async def execute(self,
@@ -74,9 +74,16 @@ class FluxKontextProNode:
mode = "single"
if isinstance(images, list):
if len(images) > 1:
print("INFO:", "Running Flux Kontext Pro multi-image edit mode.")
print("INFO:", f"{len(images)} image included for the multi-image edit.")
mode = "multi"
else:
images = images[0]
elif isinstance(images, torch.Tensor) and images.ndim == 4 and images.shape[0] > 1:
images = [images[i:i + 1] for i in range(images.shape[0])]
print("INFO:", "Running Flux Kontext Pro multi-image edit mode.")
print(
"INFO:", f"{len(images)} image included for the multi-image edit.")
print("INFO:", f"{len(images)} image included for the multi-image edit.")
mode = "multi"
else:
print("INFO:", "Running Flux Kontext Pro single-image edit mode.")
+7 -7
View File
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.flux_kontext_pro import FluxKontextProT2I
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class FluxKontextProT2INode:
@@ -18,12 +18,12 @@ class FluxKontextProT2INode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Text description of the image to generate"}),
"aspect_ratio": (["21:9", "16:9", "16:10", "4:3", "1:1", "3:4", "10:16", "9:16", "9:21"], {
"default": "1:1",
"tooltip": "The aspect ratio of the output image, ranging from \"21:9\" to \"9:21\", default is \"1:1\""
}),
"num_images": ("INT", {
"num_images": (IO.INT, {
"default": 1,
"min": 1,
"max": 4,
@@ -31,7 +31,7 @@ class FluxKontextProT2INode:
"display": "number",
"tooltip": "Number of images to generate in a single request (1 to 4)"
}),
"num_requests": ("INT", {
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -39,14 +39,14 @@ class FluxKontextProT2INode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 2.5,
"min": 0.0,
"max": 10.0,
@@ -60,7 +60,7 @@ class FluxKontextProT2INode:
RETURN_TYPES = ("IMAGE", )
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE"
CATEGORY = "UCLOUD_MODELVERSE/Flux"
FUNCTION = "execute"
async def execute(self,
+110
View File
@@ -0,0 +1,110 @@
import base64
import torch
from typing import Optional, List, Dict, Any
from comfy.comfy_types.node_typing import IO
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.gemini_flash_image import GeminiFlashImageRequest
from .modelverse_api.utils import decode_image, images2tensor
MODELS = ["gemini-3.1-flash-image", "gemini-2.5-flash-image"]
def _extract_images_from_gemini_response(resp: Dict[str, Any]) -> List[torch.Tensor]:
"""Parse Gemini API response and return list of tensors for any inline images."""
images = []
try:
candidates = resp.get("candidates", []) or []
for cand in candidates:
content = cand.get("content", {})
parts = content.get("parts", [])
for part in parts:
inline = part.get("inlineData")
if inline and isinstance(inline, dict):
data_b64 = inline.get("data")
if data_b64:
try:
img_bytes = base64.b64decode(data_b64)
pil_img = decode_image(img_bytes)
images.append(pil_img)
except Exception:
continue
except Exception:
pass
if not images:
return []
return [images2tensor(images)]
class GeminiFlashImageNode:
"""
Gemini Flash Image (text-to-image and image-edit) via Modelverse API.
Endpoint: /v1beta/models/{model}:generateContent
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "gemini-3.1-flash-image", "tooltip": "Gemini Flash Image model"}),
"prompt": (IO.STRING, {"multiline": True, "default": "Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme"}),
"mime_type": (["image/png", "image/jpeg"], {"default": "image/png"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
},
"optional": {
"image": (IO.IMAGE, {"default": None, "tooltip": "Optional input image for edit"}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Gemini"
FUNCTION = "execute"
async def execute(self,
client,
model: str,
prompt: str,
mime_type: str = "image/png",
num_requests: int = 1,
image=None):
if not prompt:
raise ValueError("Prompt is required")
mv_client = ModelverseClient(client["api_key"])
outputs: List[torch.Tensor] = []
for i in range(num_requests):
req = GeminiFlashImageRequest(prompt=prompt, model=model, image=image, mime_type=mime_type)
payload = req.build_payload()
resp = mv_client.post(req.API_PATH, payload)
if isinstance(resp, dict) and resp.get("error"):
err = resp.get("error")
raise Exception(f"GeminiFlashImage error: {err.get('message', 'Unknown error')}")
tensors = _extract_images_from_gemini_response(resp)
if not tensors:
print("WARN:", "No image data found in Gemini response; check console for details.")
continue
# tensors is a list with one batch tensor; append that tensor
outputs.append(tensors[0])
if not outputs:
# Return an empty black image tensor (1x3x1x1) to avoid breaking graph
return (torch.zeros((1, 3, 1, 1)),)
# Concatenate along batch dimension
return (torch.cat(outputs, dim=0),)
NODE_CLASS_MAPPINGS = {
"NanoBanana": GeminiFlashImageNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"NanoBanana": "Modelverse Gemini Flash Image",
}
+149
View File
@@ -0,0 +1,149 @@
"""
Gemini 3 Pro Image (Nano Banana Pro) ComfyUI Node.
Professional asset production with advanced features:
- High-resolution output (1K, 2K, 4K)
- Aspect ratio control
- Google Search grounding
- Up to 14 reference images
"""
import base64
import torch
from typing import Optional, List, Dict, Any
from comfy.comfy_types.node_typing import IO
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.gemini_pro_image import GeminiProImageRequest
from .modelverse_api.utils import decode_image, images2tensor
def _extract_images_from_gemini_response(resp: Dict[str, Any]) -> List[torch.Tensor]:
"""Extract image tensors from Gemini API response."""
tensors: List[torch.Tensor] = []
candidates = resp.get("candidates", [])
for cand in candidates:
content = cand.get("content", {})
parts = content.get("parts", [])
for part in parts:
inline = part.get("inlineData")
if inline:
data_b64 = inline.get("data")
if data_b64:
raw = base64.b64decode(data_b64)
pil_img = decode_image(raw)
tensors.append(images2tensor(pil_img))
# Also print any text response
if part.get("text"):
print("INFO: Gemini Pro Image text response:", part.get("text"))
return tensors
# Aspect ratio options
ASPECT_RATIOS = ["auto", "1:1", "2:3", "3:2", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9"]
# Resolution options
IMAGE_SIZES = ["1K", "2K", "4K"]
class GeminiProImageNode:
"""
Gemini 3 Pro Image (Nano Banana Pro) via Modelverse API.
Professional asset production with:
- High-resolution output (1K, 2K, 4K)
- Aspect ratio control
- Google Search grounding for real-time info
- Up to 14 reference images
Endpoint: /v1beta/models/gemini-3-pro-image:generateContent
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "Create a professional product photo"}),
"mime_type": (["image/png", "image/jpeg"], {"default": "image/png"}),
"aspect_ratio": (ASPECT_RATIOS, {"default": "auto"}),
"image_size": (IMAGE_SIZES, {"default": "1K"}),
"use_google_search": (IO.BOOLEAN, {"default": False, "tooltip": "Enable Google Search grounding for real-time info"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
},
"optional": {
"image1": (IO.IMAGE, {"default": None, "tooltip": "Reference image 1"}),
"image2": (IO.IMAGE, {"default": None, "tooltip": "Reference image 2"}),
"image3": (IO.IMAGE, {"default": None, "tooltip": "Reference image 3"}),
"image4": (IO.IMAGE, {"default": None, "tooltip": "Reference image 4"}),
"image5": (IO.IMAGE, {"default": None, "tooltip": "Reference image 5"}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Gemini"
FUNCTION = "execute"
async def execute(
self,
client,
prompt: str,
mime_type: str = "image/png",
aspect_ratio: str = "auto",
image_size: str = "1K",
use_google_search: bool = False,
num_requests: int = 1,
image1=None,
image2=None,
image3=None,
image4=None,
image5=None,
):
if not prompt:
raise ValueError("Prompt is required")
mv_client = ModelverseClient(client["api_key"])
# Collect input images
images = []
for img in [image1, image2, image3, image4, image5]:
if img is not None:
images.append(img)
# Process aspect_ratio
ar = aspect_ratio if aspect_ratio != "auto" else None
outputs: List[torch.Tensor] = []
for i in range(num_requests):
req = GeminiProImageRequest(
prompt=prompt,
images=images if images else None,
mime_type=mime_type,
aspect_ratio=ar,
image_size=image_size,
use_google_search=use_google_search,
)
payload = req.build_payload()
resp = mv_client.post(req.API_PATH, payload)
if isinstance(resp, dict) and resp.get("error"):
err = resp.get("error")
raise Exception(f"GeminiProImage error: {err.get('message', 'Unknown error')}")
tensors = _extract_images_from_gemini_response(resp)
if not tensors:
print("WARN:", "No image data found in Gemini Pro response; check console for details.")
continue
outputs.append(tensors[0])
if not outputs:
raise Exception("No images generated from Gemini 3 Pro Image")
return (torch.cat(outputs, dim=0),)
NODE_CLASS_MAPPINGS = {
'Gemini Pro Image (Nano Banana Pro)': GeminiProImageNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
'Gemini Pro Image (Nano Banana Pro)': 'Gemini 3 Pro Image',
}
+133
View File
@@ -0,0 +1,133 @@
import base64
import torch
from typing import List
from comfy.comfy_types.node_typing import IO
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.gpt_image_1 import GPTImage1
class GPTImage1Node:
"""
gpt-image-1 text-to-image via Modelverse /v1/images/generations API.
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower"}),
"size": (["1024x1024", "1024x1536", "1536x1024"], {"default": "1024x1024"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
},
"optional": {
"quality": (["", "low", "medium", "high"], {"default": ""}),
"output_format": (["png", "jpeg"], {"default": "png"}),
"output_compression": (IO.INT, {"default": 100, "min": 0, "max": 100, "step": 1, "display": "number"}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Gpt-img"
FUNCTION = "execute"
async def execute(
self,
client,
prompt: str,
size: str = "1024x1024",
num_requests: int = 1,
num_images: int = 1,
quality: str = "",
output_format: str = "png",
output_compression: int = 100,
):
if not prompt:
raise ValueError("Prompt is required")
mv_client = ModelverseClient(client["api_key"])
tasks = [
mv_client.async_send_request(
GPTImage1(
prompt=prompt,
num_images=num_images,
size=size,
quality=quality if quality != "" else None,
output_format=output_format,
output_compression=output_compression,
)
)
for i in range(num_requests)
]
results = await mv_client.run_tasks(tasks)
output_images_list: List[torch.Tensor] = []
for data_list in results:
if not data_list:
print("WARN:", "No output in current request. Skipping...")
continue
# Auto-detect output type: prefer URL if present, else decode b64_json
has_url = False
try:
for item in data_list:
if isinstance(item, dict) and item.get("url"):
has_url = True
break
except Exception:
has_url = False
if has_url:
try:
output_images = imageurl2tensor(data_list)
except Exception:
print("WARN:", "Failed to load URL images; attempting b64_json decode.")
has_url = False # fall through to b64 decode
if not has_url:
images = []
for item in data_list:
if not isinstance(item, dict):
continue
b64v = item.get("b64_json") or item.get("b64")
if not b64v:
continue
if isinstance(b64v, str) and b64v.startswith("data:"):
try:
b64v = b64v.split(",", 1)[1]
except Exception:
pass
try:
img_bytes = base64.b64decode(b64v)
pil_img = decode_image(img_bytes)
images.append(pil_img)
except Exception:
continue
if not images:
print("WARN:", "No decodable base64 image found.")
continue
output_images = images2tensor(images)
output_images_list.append(output_images)
if not output_images_list:
return (torch.zeros((1, 3, 1, 1)),)
return (torch.cat(output_images_list, dim=0),)
NODE_CLASS_MAPPINGS = {
"Modelverse GPTImage1Node": GPTImage1Node,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse GPTImage1Node": "Modelverse GPT Image 1",
}
+133
View File
@@ -0,0 +1,133 @@
import base64
import torch
from typing import List
from comfy.comfy_types.node_typing import IO
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.gpt_image_1_edit import GPTImage1Edit
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
class GPTImage1EditNode:
"""
gpt-image-1 image edit via /v1/images/edits (multipart/form-data).
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "Add a beach ball in the center"}),
"image": (IO.IMAGE, {"tooltip": "Input image"}),
"size": (["1024x1024", "1024x1536", "1536x1024"], {"default": "1024x1024"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
},
"optional": {
"mask": (IO.IMAGE, {"default": None, "tooltip": "Optional mask image"}),
"quality": (["", "low", "medium", "high"], {"default": ""}),
"output_format": (["png", "jpeg"], {"default": "png"}),
"output_compression": (IO.INT, {"default": 100, "min": 0, "max": 100, "step": 1, "display": "number"}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Gpt-img"
FUNCTION = "execute"
async def execute(
self,
client,
prompt: str,
image,
size: str = "1024x1024",
num_requests: int = 1,
num_images: int = 1,
mask=None,
quality: str = "",
output_format: str = "png",
output_compression: int = 100,
):
if not prompt:
raise ValueError("Prompt is required")
if image is None:
raise ValueError("Input image is required")
mv_client = ModelverseClient(client["api_key"])
tasks = [
mv_client.async_send_request(
GPTImage1Edit(
prompt=prompt,
image=image,
mask=mask,
num_images=num_images,
size=size,
quality=quality if quality != "" else None,
output_format=output_format,
output_compression=output_compression,
)
)
for _ in range(num_requests)
]
results = await mv_client.run_tasks(tasks)
output_images_list: List[torch.Tensor] = []
for data_list in results:
if not data_list:
print("WARN:", "No output in current request. Skipping...")
continue
# Auto-detect URL vs b64_json
has_url = any(isinstance(it, dict) and it.get("url") for it in data_list)
if has_url:
try:
output_images = imageurl2tensor(data_list)
except Exception:
print("WARN:", "Failed to load URL images; attempting b64_json decode.")
has_url = False
if not has_url:
images = []
for item in data_list:
if not isinstance(item, dict):
continue
b64v = item.get("b64_json") or item.get("b64")
if not b64v:
continue
if isinstance(b64v, str) and b64v.startswith("data:"):
try:
b64v = b64v.split(",", 1)[1]
except Exception:
pass
try:
img_bytes = base64.b64decode(b64v)
pil_img = decode_image(img_bytes)
images.append(pil_img)
except Exception:
continue
if not images:
print("WARN:", "No decodable base64 image found.")
continue
output_images = images2tensor(images)
output_images_list.append(output_images)
if not output_images_list:
return (torch.zeros((1, 3, 1, 1)),)
return (torch.cat(output_images_list, dim=0),)
NODE_CLASS_MAPPINGS = {
"Modelverse GPTImage1EditNode": GPTImage1EditNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse GPTImage1EditNode": "Modelverse GPT Image 1 Edit",
}
+138
View File
@@ -0,0 +1,138 @@
"""
HappyHorse 1.0 Image2Video
Model: happyhorse-1.0-i2v
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODEL = "happyhorse-1.0-i2v"
RESOLUTIONS = ["480P", "720P", "1080P"]
class HappyHorseImg2VideoNode:
"""
HappyHorse 1.0 Image2Video - image-to-video generation.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image"}),
"first_frame_url": (IO.STRING, {
"default": "",
"tooltip": "First frame image URL or base64 (use either this OR first_frame_image)",
}),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Optional text prompt to guide video generation",
}),
"resolution": (RESOLUTIONS, {
"default": "720P",
"tooltip": "Output resolution tier",
}),
"duration": (IO.INT, {
"default": 5, "min": 1, "max": 15, "step": 1,
"tooltip": "Video duration in seconds",
}),
"seed": (IO.INT, {
"default": 0, "min": 0, "max": 2147483647,
"tooltip": "Random seed (0 for random)",
}),
"watermark": (IO.BOOLEAN, {
"default": False,
"tooltip": "Add watermark to output video",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
def generate(
self,
client,
first_frame_image=None,
first_frame_url="",
prompt="",
resolution="720P",
duration=5,
seed=0,
watermark=False,
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
has_url = first_frame_url and first_frame_url.strip()
has_image = first_frame_image is not None
if has_url and has_image:
raise ValueError("Provide either first_frame_image or first_frame_url, not both")
if not has_url and not has_image:
raise ValueError("first_frame_image or first_frame_url is required for HappyHorse I2V")
mv_client = ModelverseClient(api_key)
task_input = {}
if has_url:
task_input["first_frame_url"] = first_frame_url.strip()
else:
task_input["first_frame_url"] = image_to_base64(first_frame_image)
if prompt and prompt.strip():
task_input["prompt"] = prompt.strip()
parameters = {
"resolution": resolution,
"duration": duration,
"seed": seed,
}
if watermark:
parameters["watermark"] = True
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"HappyHorse I2V task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"HappyHorse_Img2Video": HappyHorseImg2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"HappyHorse_Img2Video": "Modelverse HappyHorse I2V",
}
+156
View File
@@ -0,0 +1,156 @@
"""
HappyHorse 1.0 Reference2Video
Model: happyhorse-1.0-r2v
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODEL = "happyhorse-1.0-r2v"
RESOLUTIONS = ["480P", "720P", "1080P"]
RATIOS = ["16:9", "9:16", "1:1", "3:4", "4:3"]
class HappyHorseReference2VideoNode:
"""
HappyHorse 1.0 Reference2Video - reference-image-to-video generation.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video",
}),
},
"optional": {
"image1": (IO.IMAGE, {"tooltip": "Reference image 1"}),
"image2": (IO.IMAGE, {"tooltip": "Reference image 2"}),
"image3": (IO.IMAGE, {"tooltip": "Reference image 3"}),
"image4": (IO.IMAGE, {"tooltip": "Reference image 4"}),
"image_urls": (IO.STRING, {
"default": "",
"multiline": True,
"tooltip": "Reference image URLs, one per line",
}),
"resolution": (RESOLUTIONS, {
"default": "720P",
"tooltip": "Output resolution tier",
}),
"ratio": (RATIOS, {
"default": "16:9",
"tooltip": "Aspect ratio",
}),
"duration": (IO.INT, {
"default": 5, "min": 1, "max": 15, "step": 1,
"tooltip": "Video duration in seconds",
}),
"seed": (IO.INT, {
"default": 0, "min": 0, "max": 2147483647,
"tooltip": "Random seed (0 for random)",
}),
"watermark": (IO.BOOLEAN, {
"default": False,
"tooltip": "Add watermark to output video",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
def generate(
self,
client,
prompt,
image1=None,
image2=None,
image3=None,
image4=None,
image_urls="",
resolution="720P",
ratio="16:9",
duration=5,
seed=0,
watermark=False,
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
if not prompt or not prompt.strip():
raise ValueError("prompt is required for HappyHorse R2V")
images = []
for img in [image1, image2, image3, image4]:
if img is not None:
images.append(image_to_base64(img))
if image_urls and image_urls.strip():
for url in image_urls.strip().split("\n"):
url = url.strip()
if url:
images.append(url)
if not images:
raise ValueError("At least one reference image is required for HappyHorse R2V")
mv_client = ModelverseClient(api_key)
task_input = {
"prompt": prompt.strip(),
"images": images,
}
parameters = {
"resolution": resolution,
"ratio": ratio,
"duration": duration,
"seed": seed,
}
if watermark:
parameters["watermark"] = True
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"HappyHorse R2V task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"HappyHorse_Reference2Video": HappyHorseReference2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"HappyHorse_Reference2Video": "Modelverse HappyHorse R2V",
}
+124
View File
@@ -0,0 +1,124 @@
"""
HappyHorse 1.0 Text2Video
Model: happyhorse-1.0-t2v
"""
import time
from .modelverse_api.client import ModelverseClient
from comfy.comfy_types.node_typing import IO
MODEL = "happyhorse-1.0-t2v"
RESOLUTIONS = ["480P", "720P", "1080P"]
RATIOS = ["16:9", "9:16", "1:1", "3:4", "4:3"]
class HappyHorseText2VideoNode:
"""
HappyHorse 1.0 Text2Video - text-to-video generation.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video",
}),
},
"optional": {
"resolution": (RESOLUTIONS, {
"default": "720P",
"tooltip": "Output resolution tier",
}),
"ratio": (RATIOS, {
"default": "16:9",
"tooltip": "Aspect ratio",
}),
"duration": (IO.INT, {
"default": 5, "min": 1, "max": 15, "step": 1,
"tooltip": "Video duration in seconds",
}),
"seed": (IO.INT, {
"default": 0, "min": 0, "max": 2147483647,
"tooltip": "Random seed (0 for random)",
}),
"watermark": (IO.BOOLEAN, {
"default": False,
"tooltip": "Add watermark to output video",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/HappyHorse"
def generate(
self,
client,
prompt,
resolution="720P",
ratio="16:9",
duration=5,
seed=0,
watermark=False,
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
if not prompt or not prompt.strip():
raise ValueError("prompt is required for HappyHorse T2V")
mv_client = ModelverseClient(api_key)
task_input = {"prompt": prompt.strip()}
parameters = {
"resolution": resolution,
"ratio": ratio,
"duration": duration,
"seed": seed,
}
if watermark:
parameters["watermark"] = True
submit_res = mv_client.submit_task(MODEL, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"HappyHorse T2V task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"HappyHorse_Text2Video": HappyHorseText2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"HappyHorse_Text2Video": "Modelverse HappyHorse T2V",
}
+167
View File
@@ -0,0 +1,167 @@
"""
Kling V3 - Unified text/image-to-video and motion control model
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.kling_common import (
ASPECT_RATIOS,
CHARACTER_ORIENTATIONS,
KLING_V3_TYPES,
MODEL_KLING_V3,
MODES,
SOUNDS,
YES_NO,
)
from .modelverse_api.requests.kling_v3 import KlingV3
from comfy.comfy_types.node_typing import IO
class KlingV3Node:
"""
Kling V3 unified video generation.
Auto-routes to text-to-video, image-to-video, or motion control.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video",
}),
},
"optional": {
"negative_prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Negative prompt to avoid unwanted content",
}),
"kling_v3_type": (KLING_V3_TYPES, {
"default": "auto",
"tooltip": "Task type; auto infers from inputs (image→i2v, video→motion_control)",
}),
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image (image-to-video / motion control)"}),
"first_frame_url": (IO.STRING, {
"default": "",
"tooltip": "First frame URL (use either this OR first_frame_image, not both)",
}),
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image (image-to-video)"}),
"last_frame_url": (IO.STRING, {
"default": "",
"tooltip": "Last frame URL (use either this OR last_frame_image, not both)",
}),
"reference_video_url": (IO.STRING, {
"default": "",
"tooltip": "Reference video URL (motion control; auto-selects motion_control when set)",
}),
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "Output aspect ratio"}),
"duration": (IO.INT, {
"default": 5, "min": 3, "max": 15, "step": 1,
"tooltip": "Video duration in seconds (3–15)",
}),
"mode": (MODES, {"default": "std", "tooltip": "std: 720p, pro: 1080p"}),
"sound": (SOUNDS, {"default": "off", "tooltip": "Generate synchronized audio"}),
"shot_type": (IO.STRING, {
"default": "",
"tooltip": "Optional shot type, e.g. 'multi' for multi-shot",
}),
"character_orientation": (CHARACTER_ORIENTATIONS, {
"default": "image",
"tooltip": "Motion control: follow image or video orientation",
}),
"keep_original_sound": (YES_NO, {
"default": "no",
"tooltip": "Motion control: keep original reference video sound",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Kling"
def generate(
self,
client,
prompt,
negative_prompt="",
kling_v3_type="auto",
first_frame_image=None,
first_frame_url="",
last_frame_image=None,
last_frame_url="",
reference_video_url="",
aspect_ratio="16:9",
duration=5,
mode="std",
sound="off",
shot_type="",
character_orientation="image",
keep_original_sound="no",
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
request = KlingV3(
prompt=prompt,
negative_prompt=negative_prompt,
kling_v3_type=kling_v3_type,
first_frame=first_frame_image,
first_frame_url=first_frame_url,
last_frame=last_frame_image,
last_frame_url=last_frame_url,
reference_video_url=reference_video_url,
aspect_ratio=aspect_ratio,
duration=duration,
mode=mode,
sound=sound,
shot_type=shot_type,
character_orientation=character_orientation,
keep_original_sound=keep_original_sound,
)
mv_client = ModelverseClient(api_key)
print(f"Submitting Kling V3 task: model={MODEL_KLING_V3}, type={kling_v3_type}, prompt={prompt!r}")
submit_res = mv_client.submit_task_request(request)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Kling V3 task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Kling_V3": KlingV3Node,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Kling_V3": "Modelverse Kling V3",
}
+166
View File
@@ -0,0 +1,166 @@
"""
Kling V3 Omni - Multimodal video generation and editing model
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.kling_common import (
ASPECT_RATIOS,
MODEL_KLING_V3_OMNI,
MODES,
REFER_TYPES,
SOUNDS,
YES_NO,
)
from .modelverse_api.requests.kling_v3_omni import KlingV3Omni
from comfy.comfy_types.node_typing import IO
class KlingV3OmniNode:
"""
Kling V3 Omni multimodal video generation.
Supports reference images, videos, and element library IDs.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video",
}),
},
"optional": {
"negative_prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Negative prompt to avoid unwanted content",
}),
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame reference image"}),
"first_frame_url": (IO.STRING, {
"default": "",
"tooltip": "First frame URL (use either this OR first_frame_image, not both)",
}),
"last_frame_image": (IO.IMAGE, {"tooltip": "End frame reference image"}),
"last_frame_url": (IO.STRING, {
"default": "",
"tooltip": "End frame URL (use either this OR last_frame_image, not both)",
}),
"reference_video_url": (IO.STRING, {
"default": "",
"tooltip": "Reference video URL for editing or style reference",
}),
"refer_type": (REFER_TYPES, {
"default": "feature",
"tooltip": "feature: style/motion reference; base: video editing",
}),
"keep_original_sound": (YES_NO, {
"default": "no",
"tooltip": "Keep original audio from reference video",
}),
"element_id": (IO.INT, {
"default": 0, "min": 0,
"tooltip": "Kling element library ID (0 to skip)",
}),
"aspect_ratio": (ASPECT_RATIOS, {
"default": "16:9",
"tooltip": "Output aspect ratio (required without first-frame or video editing)",
}),
"duration": (IO.INT, {
"default": 5, "min": 3, "max": 15, "step": 1,
"tooltip": "Video duration in seconds (3–15)",
}),
"mode": (MODES, {"default": "std", "tooltip": "std: 720p, pro: 1080p"}),
"sound": (SOUNDS, {
"default": "off",
"tooltip": "Generate synchronized audio (must be off when reference_video_url is set)",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Kling"
def generate(
self,
client,
prompt,
negative_prompt="",
first_frame_image=None,
first_frame_url="",
last_frame_image=None,
last_frame_url="",
reference_video_url="",
refer_type="feature",
keep_original_sound="no",
element_id=0,
aspect_ratio="16:9",
duration=5,
mode="std",
sound="off",
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
request = KlingV3Omni(
prompt=prompt,
negative_prompt=negative_prompt,
first_frame=first_frame_image,
first_frame_url=first_frame_url,
last_frame=last_frame_image,
last_frame_url=last_frame_url,
reference_video_url=reference_video_url,
refer_type=refer_type,
keep_original_sound=keep_original_sound,
element_id=element_id,
aspect_ratio=aspect_ratio,
duration=duration,
mode=mode,
sound=sound,
)
mv_client = ModelverseClient(api_key)
print(f"Submitting Kling V3 Omni task: model={MODEL_KLING_V3_OMNI}, prompt={prompt!r}")
submit_res = mv_client.submit_task_request(request)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Kling V3 Omni task submitted: {task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Kling_V3_Omni": KlingV3OmniNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Kling_V3_Omni": "Modelverse Kling V3 Omni",
}
+56 -46
View File
@@ -4,47 +4,43 @@ import asyncio
from .utils import BaseRequest
import asyncio
from .utils import BaseRequest
class ModelverseClient:
"""
UCloud Modelverse API Client
This class handles the core communication with the Modelverse API.
"""
BASE_URL = "https://api.modelverse.cn"
API_PATH = "/v1/images/generations"
def __init__(self, api_key):
"""
Initialize Modelverse API client
Args:
api_key (str): Modelverse API key
"""
self.api_key = api_key
self.headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
self.headers = {"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"}
def post(self, endpoint, payload, timeout=180):
url = f"{self.BASE_URL}{endpoint}"
response = requests.post(url, headers=self.headers, json=payload, timeout=timeout)
return self._handle_response(response)
def post(self, payload, timeout=180):
"""
Send POST request to Modelverse API
def post_multipart(self, endpoint, data=None, files=None, timeout=180):
"""POST with multipart/form-data. Content-Type is set by requests automatically."""
url = f"{self.BASE_URL}{endpoint}"
# Do not set Content-Type explicitly when using files; requests will handle it.
headers = {k: v for k, v in self.headers.items() if k.lower() != "content-type"}
response = requests.post(url, headers=headers, data=data, files=files, timeout=timeout)
return self._handle_response(response)
Args:
endpoint (str): API endpoint
payload (dict): Request payload
timeout (float, optional): Request timeout in seconds
Returns:
dict: API response
"""
url = f"{self.BASE_URL}{self.API_PATH}"
response = requests.post(
url, headers=self.headers, json=payload, timeout=timeout)
def get(self, endpoint, params=None, timeout=180):
url = f"{self.BASE_URL}{endpoint}"
headers = {"Authorization": f"Bearer {self.api_key}"}
response = requests.get(url, headers=headers, params=params, timeout=timeout)
return self._handle_response(response)
def _handle_response(self, response):
if response.status_code == 401:
raise Exception("Unauthorized: Invalid API key")
# For backward compatibility with older error formats
if response.status_code != 200:
error_message = f"Error: {response.status_code}"
try:
@@ -60,26 +56,40 @@ class ModelverseClient:
if response_data['code'] == 401:
raise Exception("Unauthorized: Invalid API key")
if response_data['code'] != 200:
raise Exception(
f"API Error: {response_data.get('message', 'Unknown error')}")
raise Exception(f"API Error: {response_data.get('message', 'Unknown error')}")
return response_data.get('data', {})
return response_data
# --- New Methods for T2V ---
def submit_task(self, model, task_input, parameters):
endpoint = "/v1/tasks/submit"
payload = {
"model": model,
"input": task_input,
"parameters": parameters
}
return self.post(endpoint, payload)
def submit_task_request(self, request: BaseRequest):
return self.post(request.API_PATH, request.build_payload())
def get_task_status(self, task_id):
endpoint = f"/v1/tasks/status"
params = {"task_id": task_id}
return self.get(endpoint, params=params)
# --- Restored Async Methods for existing nodes ---
async def async_send_request(self, request: BaseRequest):
"""
Sends an API request using a request object.
Args:
request (BaseRequest): The request object containing payload and endpoint logic.
Returns:
dict: API response or task result.
"""
payload = request.build_payload()
if "seed" in payload:
payload["seed"] = payload["seed"] % 2147483647 if payload["seed"] != -1 else -1
response = self.post(payload)
endpoint = request.API_PATH
# Support multipart form requests when available
if hasattr(request, "build_multipart") and callable(getattr(request, "build_multipart")):
data, files = request.build_multipart()
response = self.post_multipart(endpoint, data=data, files=files)
else:
payload = request.build_payload()
if isinstance(payload, dict) and "seed" in payload:
payload["seed"] = payload["seed"] % 2147483647 if payload["seed"] != -1 else -1
response = self.post(endpoint, payload)
return response.get("data", [])
async def run_tasks(self, tasks):
@@ -0,0 +1,198 @@
"""
Doubao Seedance 2.0 task submit request builder.
API: https://www.compshare.cn/docs/modelverse/models/video_api/doubao-seedance-2-0
"""
from typing import Any, Dict, List, Optional
from pydantic import Field
from torch import Tensor
from ..utils import BaseRequest, image_to_base64
MODEL = "doubao-seedance-2-0-260128"
RESOLUTIONS = ["480p", "720p", "1080p"]
RATIOS = ["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"]
def _resolve_image(image: Optional[Tensor], url: str, label: str) -> Optional[str]:
has_url = url and url.strip()
has_image = image is not None
if has_url and has_image:
raise ValueError(f"{label}: provide either image or url, not both")
if has_url:
return url.strip()
if has_image:
encoded = image_to_base64(image)
if not encoded:
raise ValueError(f"{label}: failed to convert image to base64")
return encoded
return None
def _normalize_prompt(value) -> str:
if value is None:
return ""
if isinstance(value, list):
value = value[0] if value else ""
if isinstance(value, dict):
return ""
return str(value).strip()
def _image_content(url: str, role: Optional[str] = None) -> Dict[str, Any]:
item: Dict[str, Any] = {
"type": "image_url",
"image_url": {"url": url},
}
if role:
item["role"] = role
return item
class DoubaoSeedance2(BaseRequest):
"""
Doubao Seedance 2.0 video generation task request.
Supports text, first/last frame images, reference image/video/audio.
"""
API_PATH = "/v1/tasks/submit"
prompt: str = Field(default="", description="Text prompt for video generation")
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
first_frame_url: str = Field(default="", description="First frame image URL or base64")
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
reference_image: Optional[Tensor] = Field(default=None, description="Reference image tensor")
reference_image_url: str = Field(default="", description="Reference image URL or base64")
reference_video_url: str = Field(default="", description="Reference video URL")
reference_audio_url: str = Field(default="", description="Reference audio URL")
duration: int = Field(default=5, ge=4, le=15, description="Video duration in seconds")
resolution: str = Field(default="720p", description="Output resolution")
ratio: str = Field(default="adaptive", description="Aspect ratio")
seed: int = Field(default=0, ge=0, le=2147483647, description="Random seed")
generate_audio: bool = Field(default=False, description="Whether to generate synchronized audio")
camera_fixed: bool = Field(default=False, description="Whether to fix camera position")
watermark: bool = Field(default=False, description="Whether to add watermark")
draft: bool = Field(default=False, description="Draft mode (480p only)")
def __init__(
self,
prompt: str = "",
first_frame: Optional[Tensor] = None,
first_frame_url: str = "",
last_frame: Optional[Tensor] = None,
last_frame_url: str = "",
reference_image: Optional[Tensor] = None,
reference_image_url: str = "",
reference_video_url: str = "",
reference_audio_url: str = "",
duration: int = 5,
resolution: str = "720p",
ratio: str = "adaptive",
seed: int = 0,
generate_audio: bool = False,
camera_fixed: bool = False,
watermark: bool = False,
draft: bool = False,
**kwargs,
):
super().__init__(**kwargs)
self.prompt = _normalize_prompt(prompt)
self.first_frame = first_frame
self.first_frame_url = first_frame_url
self.last_frame = last_frame
self.last_frame_url = last_frame_url
self.reference_image = reference_image
self.reference_image_url = reference_image_url
self.reference_video_url = reference_video_url
self.reference_audio_url = reference_audio_url
self.duration = duration
self.resolution = resolution
self.ratio = ratio
self.seed = seed
self.generate_audio = generate_audio
self.camera_fixed = camera_fixed
self.watermark = watermark
self.draft = draft
def build_content(self) -> List[Dict[str, Any]]:
if self.draft and self.resolution != "480p":
raise ValueError("Draft mode only supports 480p resolution")
first_url = _resolve_image(self.first_frame, self.first_frame_url, "First frame")
last_url = _resolve_image(self.last_frame, self.last_frame_url, "Last frame")
ref_img_url = _resolve_image(self.reference_image, self.reference_image_url, "Reference image")
content: List[Dict[str, Any]] = []
if self.prompt and self.prompt.strip():
content.append({"type": "text", "text": self.prompt.strip()})
frame_images = [(first_url, "first_frame"), (last_url, "last_frame")]
frame_images = [(url, role) for url, role in frame_images if url]
if ref_img_url:
frame_images.append((ref_img_url, "reference_image"))
for url, role in frame_images:
content.append(_image_content(url, role))
if self.reference_video_url and self.reference_video_url.strip():
content.append({
"type": "video_url",
"video_url": {"url": self.reference_video_url.strip()},
"role": "reference_video",
})
if self.reference_audio_url and self.reference_audio_url.strip():
content.append({
"type": "audio_url",
"audio_url": {"url": self.reference_audio_url.strip()},
"role": "reference_audio",
})
if not content:
raise ValueError(
"At least a text prompt or one input asset (image/video/audio) is required. "
f"Current prompt is empty (received: {self.prompt!r})"
)
return content
def build_parameters(self) -> Dict[str, Any]:
return {
"duration": self.duration,
"resolution": self.resolution,
"ratio": self.ratio,
"seed": self.seed,
"generate_audio": self.generate_audio,
"camera_fixed": self.camera_fixed,
"watermark": self.watermark,
"draft": self.draft,
}
def build_payload(self) -> Dict[str, Any]:
return {
"model": MODEL,
"input": {"content": self.build_content()},
"parameters": self.build_parameters(),
}
def field_required(self):
return []
def field_order(self):
return [
"prompt",
"first_frame",
"last_frame",
"reference_image",
"reference_video_url",
"reference_audio_url",
"duration",
"resolution",
"ratio",
"seed",
"generate_audio",
"camera_fixed",
"watermark",
"draft",
]
+1
View File
@@ -8,6 +8,7 @@ class FluxDev(BaseRequest):
"""
Flux-dev text to image model, 12 billion parameter rectified flow transformer
"""
API_PATH = "/v1/images/generations"
guidance_scale: Optional[float] = Field(
default=2.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
num_images: Optional[int] = Field(
@@ -8,6 +8,7 @@ class FluxKontextMax(BaseRequest):
"""
Flux Kontext Max for image editing.
"""
API_PATH = "/v1/images/generations"
guidance_scale: Optional[float] = Field(
default=2.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
image: Tensor = Field(...,
@@ -60,6 +61,7 @@ class FluxKontextMaxMulti(BaseRequest):
"""
Flux Kontext Max for multiple image inputs.
"""
API_PATH = "/v1/images/generations"
guidance_scale: Optional[float] = Field(
default=3.5, description="The CFG (Classifier Free Guidance) scale is a measure of how close you want the model to stick to your prompt when looking for a related image to show you.")
images: list[Tensor] = Field(...,
@@ -112,6 +114,7 @@ class FluxKontextMaxT2I(BaseRequest):
"""
Flux Kontext Pro text-to-image model
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(...,
description="The prompt to generate an image from.")
aspect_ratio: Optional[str] = Field(
@@ -8,6 +8,7 @@ class FluxKontextPro(BaseRequest):
"""
Flux Kontext Pro for image editing.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(...,
description="The prompt to generate an image from.")
image: Tensor = Field(...,
@@ -61,6 +62,7 @@ class FluxKontextProMulti(BaseRequest):
"""
Flux Kontext Pro for multiple image inputs.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(...,
description="The prompt to generate an image from.")
images: list[Tensor] = Field(...,
@@ -113,6 +115,7 @@ class FluxKontextProT2I(BaseRequest):
"""
Flux Kontext Pro text-to-image model
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(...,
description="The prompt to generate an image from.")
aspect_ratio: Optional[str] = Field(
@@ -0,0 +1,83 @@
from typing import Optional, List, Dict, Any
from pydantic import Field
from ..utils import BaseRequest
from torch import Tensor
import base64
import io
from PIL import Image
import numpy as np
def _tensor_to_base64(image: Tensor, mime_type: str = "image/png") -> Dict[str, str]:
"""Convert a ComfyUI image tensor (HWC or 4D with batch) to base64 without data URI."""
if image is None:
return None
# If batch, take first frame
if hasattr(image, 'shape') and len(image.shape) == 4:
image = image[0]
# Tensor (H, W, C) in 0..1 -> uint8
np_img = np.clip(255.0 * image.cpu().numpy(), 0, 255).astype(np.uint8)
pil_img = Image.fromarray(np_img)
fmt = 'PNG' if mime_type.lower().endswith('png') else 'JPEG'
with io.BytesIO() as bio:
pil_img.save(bio, format=fmt)
data = bio.getvalue()
return {
"mimeType": mime_type,
"data": base64.b64encode(data).decode("utf-8"),
}
class GeminiFlashImageRequest(BaseRequest):
"""
Request builder for Gemini Flash Image generateContent endpoint.
Supports text-to-image and image-edit (text+image to image).
"""
API_PATH_TEMPLATE = "/v1beta/models/{model}:generateContent"
prompt: str = Field(..., description="Text prompt")
model: str = Field(default="gemini-3.1-flash-image", description="Gemini Flash Image model")
image: Optional[Tensor] = Field(default=None, description="Optional input image for edit")
mime_type: str = Field(default="image/png", description="MIME type for inline image data")
def __init__(self, prompt: str, model: str = "gemini-3.1-flash-image", image: Optional[Tensor] = None, mime_type: str = "image/png", **kwargs):
super().__init__(**kwargs)
self.prompt = prompt
self.model = model
self.API_PATH = self.API_PATH_TEMPLATE.format(model=model)
self.image = image
self.mime_type = mime_type
def build_payload(self) -> Dict[str, Any]:
parts: List[Dict[str, Any]] = []
if self.prompt:
parts.append({"text": self.prompt})
if isinstance(self.image, Tensor):
parts.append({
"inlineData": _tensor_to_base64(self.image, self.mime_type)
})
payload = {
"contents": [
{
"role": "user",
"parts": parts,
}
],
"generationConfig": {
"responseModalities": ["TEXT", "IMAGE"],
},
}
return self._remove_empty_fields(payload)
def field_required(self):
return ["prompt"]
def field_order(self):
return ["model", "prompt", "image", "mime_type"]
@@ -0,0 +1,121 @@
"""
Gemini 3 Pro Image (Nano Banana Pro) request builder.
Supports text-to-image and image editing with advanced features like
aspect ratio, resolution control, and Google Search grounding.
"""
import io
import base64
import numpy as np
from PIL import Image
from typing import Optional, List, Dict, Any
from pydantic import Field
from torch import Tensor
from ..utils import BaseRequest
def _tensor_to_base64(image: Tensor, mime_type: str = "image/png") -> Dict[str, str]:
"""Convert a ComfyUI image tensor (HWC or 4D with batch) to base64 without data URI."""
if image is None:
return None
# If batch, take first frame
if hasattr(image, 'shape') and len(image.shape) == 4:
image = image[0]
# Tensor (H, W, C) in 0..1 -> uint8
np_img = np.clip(255.0 * image.cpu().numpy(), 0, 255).astype(np.uint8)
pil_img = Image.fromarray(np_img)
fmt = 'PNG' if mime_type.lower().endswith('png') else 'JPEG'
with io.BytesIO() as bio:
pil_img.save(bio, format=fmt)
data = bio.getvalue()
return {
"mimeType": mime_type,
"data": base64.b64encode(data).decode("utf-8"),
}
class GeminiProImageRequest(BaseRequest):
"""
Request builder for Gemini 3 Pro Image (Nano Banana Pro) generateContent endpoint.
Supports text-to-image, image editing, and advanced features like aspect ratio,
resolution control (1K/2K/4K), and Google Search grounding.
"""
API_PATH = "/v1beta/models/gemini-3-pro-image:generateContent"
prompt: str = Field(..., description="Text prompt")
images: Optional[List[Tensor]] = Field(default=None, description="Optional input images (up to 14)")
mime_type: str = Field(default="image/png", description="MIME type for inline image data")
aspect_ratio: Optional[str] = Field(default=None, description="Aspect ratio: 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9")
image_size: Optional[str] = Field(default=None, description="Image resolution: 1K, 2K, 4K")
use_google_search: bool = Field(default=False, description="Enable Google Search grounding")
def __init__(
self,
prompt: str,
images: Optional[List[Tensor]] = None,
mime_type: str = "image/png",
aspect_ratio: Optional[str] = None,
image_size: Optional[str] = None,
use_google_search: bool = False,
**kwargs
):
super().__init__(**kwargs)
self.prompt = prompt
self.images = images
self.mime_type = mime_type
self.aspect_ratio = aspect_ratio
self.image_size = image_size
self.use_google_search = use_google_search
def build_payload(self) -> Dict[str, Any]:
parts: List[Dict[str, Any]] = []
# Add text prompt
if self.prompt:
parts.append({"text": self.prompt})
# Add images (up to 14)
if self.images:
for img in self.images[:14]: # Limit to 14 images
if isinstance(img, Tensor):
parts.append({
"inlineData": _tensor_to_base64(img, self.mime_type)
})
payload = {
"contents": [
{
"role": "user",
"parts": parts,
}
],
"generationConfig": {
"responseModalities": ["TEXT", "IMAGE"],
},
}
# Add imageConfig if aspect_ratio or image_size specified
if self.aspect_ratio or self.image_size:
image_config = {}
if self.aspect_ratio:
image_config["aspectRatio"] = self.aspect_ratio
if self.image_size:
image_config["imageSize"] = self.image_size
payload["generationConfig"]["imageConfig"] = image_config
# Add Google Search tool if enabled
if self.use_google_search:
payload["tools"] = [{"google_search": {}}]
return self._remove_empty_fields(payload)
def field_required(self):
return ["prompt"]
def field_order(self):
return ["prompt", "images", "mime_type", "aspect_ratio", "image_size", "use_google_search"]
+65
View File
@@ -0,0 +1,65 @@
from typing import Optional
from pydantic import Field
from ..utils import BaseRequest
class GPTImage1(BaseRequest):
"""
gpt-image-1 text-to-image via /v1/images/generations.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(..., description="Prompt text")
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
size: Optional[str] = Field(default="1024x1024", description="Output size: 1024x1024, 1024x1536, 1536x1024")
quality: Optional[str] = Field(default=None, description="Image quality: low, medium, high")
output_format: Optional[str] = Field(default="png", description="Output format: png or jpeg")
output_compression: Optional[int] = Field(default=100, ge=0, le=100, description="Compression strength 0-100")
# negative_prompt and response_format removed per API behavior
def __init__(
self,
prompt: str,
num_images: Optional[int] = 1,
size: Optional[str] = "1024x1024",
quality: Optional[str] = None,
output_format: Optional[str] = "png",
output_compression: Optional[int] = 100,
# guidance_scale removed per API behavior
# negative_prompt/response_format removed per API behavior
**kwargs,
):
super().__init__(**kwargs)
self.prompt = prompt
self.num_images = num_images
self.size = size
self.quality = quality
self.output_format = output_format
self.output_compression = output_compression
def build_payload(self) -> dict:
payload = {
"model": "gpt-image-1",
"prompt": self.prompt,
"n": self.num_images,
"size": self.size,
"quality": self.quality,
"output_format": self.output_format,
"output_compression": self.output_compression,
}
return self._remove_empty_fields(payload)
def field_required(self):
return ["prompt"]
def field_order(self):
return [
"model",
"prompt",
"n",
"size",
"quality",
"output_format",
"output_compression",
]
@@ -0,0 +1,115 @@
from typing import Optional, Tuple, Dict, Any
from pydantic import Field
from ..utils import BaseRequest, tensor2images
from torch import Tensor
import io
from PIL import Image
def _tensor_to_png_file(image: Tensor, filename: str) -> Tuple[str, bytes, str] | None:
"""Convert a ComfyUI tensor image (3D or 4D) to a PNG file tuple for requests files."""
if image is None:
return None
# Keep batch if present; tensor2images expects a batch dimension
try:
if hasattr(image, 'shape') and len(image.shape) == 3:
# add batch dim
img_batch = image.unsqueeze(0)
else:
img_batch = image
pil_img = tensor2images(img_batch)[0]
except Exception:
# Fallback: attempt naive conversion
from PIL import Image as _Image
import numpy as _np
arr = (image.cpu().numpy() * 255.0).clip(0, 255).astype('uint8')
pil_img = _Image.fromarray(arr)
with io.BytesIO() as bio:
pil_img.save(bio, format="PNG")
data = bio.getvalue()
return (filename, data, "image/png")
class GPTImage1Edit(BaseRequest):
"""
gpt-image-1 image edit via /v1/images/edits using multipart/form-data.
"""
API_PATH = "/v1/images/edits"
prompt: str = Field(..., description="Edit prompt")
image: Tensor = Field(..., description="Input image")
mask: Optional[Tensor] = Field(default=None, description="Optional mask image")
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
size: Optional[str] = Field(default="1024x1024", description="Output size: 1024x1024, 1024x1536, 1536x1024")
quality: Optional[str] = Field(default=None, description="Image quality: low, medium, high")
output_format: Optional[str] = Field(default="png", description="Output format: png or jpeg")
output_compression: Optional[int] = Field(default=100, ge=0, le=100, description="Compression strength 0-100")
def __init__(
self,
prompt: str,
image: Tensor,
mask: Optional[Tensor] = None,
num_images: Optional[int] = 1,
size: Optional[str] = "1024x1024",
quality: Optional[str] = None,
output_format: Optional[str] = "png",
output_compression: Optional[int] = 100,
**kwargs,
):
super().__init__(**kwargs)
self.prompt = prompt
self.image = image
self.mask = mask
self.num_images = num_images
self.size = size
self.quality = quality
self.output_format = output_format
self.output_compression = output_compression
def build_multipart(self) -> Tuple[Dict[str, Any], Dict[str, Any]]:
data = {
"model": "gpt-image-1",
"prompt": self.prompt,
"n": self.num_images,
"size": self.size,
"output_format": self.output_format,
"output_compression": str(self.output_compression) if self.output_compression is not None else None,
}
if self.quality:
data["quality"] = self.quality
# Filter None values
data = {k: v for k, v in data.items() if v is not None and v != ""}
files = {}
img_file = _tensor_to_png_file(self.image, "image.png")
if img_file is not None:
files["image"] = img_file
if self.mask is not None:
mask_file = _tensor_to_png_file(self.mask, "mask.png")
if mask_file is not None:
files["mask"] = mask_file
return data, files
# For interface completeness; not used by multipart path
def build_payload(self) -> dict:
return {}
def field_required(self):
return ["prompt", "image"]
def field_order(self):
return [
"model",
"prompt",
"image",
"mask",
"n",
"size",
"quality",
"output_format",
"output_compression",
]
@@ -0,0 +1,52 @@
"""Shared helpers for Kling V3 request builders."""
import base64
from typing import Optional
from torch import Tensor
from ..utils import encode_image, tensor2images
MODEL_KLING_V3 = "kling-v3"
MODEL_KLING_V3_OMNI = "kling-v3-omni"
MODES = ["std", "pro"]
SOUNDS = ["off", "on"]
ASPECT_RATIOS = ["16:9", "9:16", "1:1"]
KLING_V3_TYPES = ["auto", "t2v", "i2v", "motion_control"]
CHARACTER_ORIENTATIONS = ["image", "video"]
YES_NO = ["yes", "no"]
REFER_TYPES = ["feature", "base"]
def normalize_kling_image_value(value: str) -> str:
"""Kling accepts http(s) URLs or raw base64 without a data:image prefix."""
value = value.strip()
if value.startswith("data:image/") and "," in value:
return value.split(",", 1)[1]
return value
def resolve_image(image: Optional[Tensor], url: str, label: str) -> Optional[str]:
has_url = url and url.strip()
has_image = image is not None
if has_url and has_image:
raise ValueError(f"{label}: provide either image or url, not both")
if has_url:
return normalize_kling_image_value(url)
if has_image:
data_bytes, _ = encode_image(tensor2images(image)[0])
if not data_bytes:
raise ValueError(f"{label}: failed to convert image to base64")
return base64.b64encode(data_bytes).decode("utf-8")
return None
def normalize_prompt(value) -> str:
if value is None:
return ""
if isinstance(value, list):
value = value[0] if value else ""
if isinstance(value, dict):
return ""
return str(value).strip()
+172
View File
@@ -0,0 +1,172 @@
"""
Kling V3 task submit request builder.
Routes to t2v / i2v / motion_control via input fields or parameters.kling_v3_type.
"""
from typing import Any, Dict, Optional
from pydantic import Field
from torch import Tensor
from ..utils import BaseRequest
from .kling_common import (
ASPECT_RATIOS,
CHARACTER_ORIENTATIONS,
KLING_V3_TYPES,
MODEL_KLING_V3,
MODES,
SOUNDS,
YES_NO,
normalize_prompt,
resolve_image,
)
class KlingV3(BaseRequest):
"""
Kling V3 unified video generation task request.
Auto-routes to text-to-video, image-to-video, or motion control.
"""
API_PATH = "/v1/tasks/submit"
prompt: str = Field(default="", description="Text prompt for video generation")
negative_prompt: str = Field(default="", description="Negative prompt")
kling_v3_type: str = Field(
default="auto",
description="Task variant: auto, t2v, i2v, motion_control",
)
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
first_frame_url: str = Field(default="", description="First frame image URL or base64")
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
reference_video_url: str = Field(default="", description="Reference video URL for motion control")
aspect_ratio: str = Field(default="16:9", description="Aspect ratio")
duration: int = Field(default=5, ge=3, le=15, description="Video duration in seconds")
mode: str = Field(default="std", description="Generation mode: std or pro")
sound: str = Field(default="off", description="Generate synchronized audio: on or off")
shot_type: str = Field(default="", description="Shot type, e.g. multi for multi-shot")
character_orientation: str = Field(
default="image",
description="Motion control orientation: image or video",
)
keep_original_sound: str = Field(
default="no",
description="Keep original sound from reference video: yes or no",
)
def __init__(
self,
prompt: str = "",
negative_prompt: str = "",
kling_v3_type: str = "auto",
first_frame: Optional[Tensor] = None,
first_frame_url: str = "",
last_frame: Optional[Tensor] = None,
last_frame_url: str = "",
reference_video_url: str = "",
aspect_ratio: str = "16:9",
duration: int = 5,
mode: str = "std",
sound: str = "off",
shot_type: str = "",
character_orientation: str = "image",
keep_original_sound: str = "no",
**kwargs,
):
super().__init__(**kwargs)
self.prompt = normalize_prompt(prompt)
self.negative_prompt = normalize_prompt(negative_prompt)
self.kling_v3_type = kling_v3_type
self.first_frame = first_frame
self.first_frame_url = first_frame_url
self.last_frame = last_frame
self.last_frame_url = last_frame_url
self.reference_video_url = reference_video_url
self.aspect_ratio = aspect_ratio
self.duration = duration
self.mode = mode
self.sound = sound
self.shot_type = shot_type
self.character_orientation = character_orientation
self.keep_original_sound = keep_original_sound
def _is_motion_control(self) -> bool:
if self.kling_v3_type == "motion_control":
return True
if self.kling_v3_type == "auto" and self.reference_video_url and self.reference_video_url.strip():
return True
return False
def build_input(self) -> Dict[str, Any]:
task_input: Dict[str, Any] = {}
if self.prompt:
task_input["prompt"] = self.prompt
if self.negative_prompt:
task_input["negative_prompt"] = self.negative_prompt
first_url = resolve_image(self.first_frame, self.first_frame_url, "First frame")
last_url = resolve_image(self.last_frame, self.last_frame_url, "Last frame")
if self._is_motion_control():
if not first_url:
raise ValueError("Motion control requires a first frame image or URL")
if not self.reference_video_url or not self.reference_video_url.strip():
raise ValueError("Motion control requires reference_video_url")
task_input["first_frame_url"] = first_url
task_input["video_url"] = self.reference_video_url.strip()
return task_input
if first_url:
task_input["first_frame_url"] = first_url
if last_url:
task_input["last_frame_url"] = last_url
if not task_input.get("prompt") and not first_url:
raise ValueError(
"At least a prompt or first frame image is required. "
f"Current prompt is empty (received: {self.prompt!r})"
)
return task_input
def build_parameters(self) -> Dict[str, Any]:
params: Dict[str, Any] = {
"duration": self.duration,
"aspect_ratio": self.aspect_ratio,
"mode": self.mode,
"sound": self.sound,
}
if self.kling_v3_type and self.kling_v3_type != "auto":
params["kling_v3_type"] = self.kling_v3_type
if self.shot_type:
params["shot_type"] = self.shot_type
if self._is_motion_control():
params["character_orientation"] = self.character_orientation
params["keep_original_sound"] = self.keep_original_sound
return params
def build_payload(self) -> Dict[str, Any]:
return {
"model": MODEL_KLING_V3,
"input": self.build_input(),
"parameters": self.build_parameters(),
}
def field_required(self):
return []
def field_order(self):
return [
"prompt",
"negative_prompt",
"kling_v3_type",
"first_frame",
"last_frame",
"reference_video_url",
"aspect_ratio",
"duration",
"mode",
"sound",
"shot_type",
"character_orientation",
"keep_original_sound",
]
+155
View File
@@ -0,0 +1,155 @@
"""
Kling V3 Omni task submit request builder.
Uses image_list / video_list / element_list parameters.
"""
from typing import Any, Dict, List, Optional
from pydantic import Field
from torch import Tensor
from ..utils import BaseRequest
from .kling_common import (
ASPECT_RATIOS,
MODEL_KLING_V3_OMNI,
MODES,
REFER_TYPES,
SOUNDS,
YES_NO,
normalize_prompt,
resolve_image,
)
class KlingV3Omni(BaseRequest):
"""
Kling V3 Omni multimodal video generation task request.
Supports reference images, videos, and element library IDs.
"""
API_PATH = "/v1/tasks/submit"
prompt: str = Field(default="", description="Text prompt for video generation")
negative_prompt: str = Field(default="", description="Negative prompt")
first_frame: Optional[Tensor] = Field(default=None, description="First frame image tensor")
first_frame_url: str = Field(default="", description="First frame image URL or base64")
last_frame: Optional[Tensor] = Field(default=None, description="Last frame image tensor")
last_frame_url: str = Field(default="", description="Last frame image URL or base64")
reference_video_url: str = Field(default="", description="Reference video URL")
refer_type: str = Field(default="feature", description="Video reference type: feature or base")
keep_original_sound: str = Field(default="no", description="Keep original video sound: yes or no")
element_id: int = Field(default=0, description="Element library ID (0 to skip)")
aspect_ratio: str = Field(default="16:9", description="Aspect ratio")
duration: int = Field(default=5, ge=3, le=15, description="Video duration in seconds")
mode: str = Field(default="std", description="Generation mode: std or pro")
sound: str = Field(default="off", description="Generate synchronized audio: on or off")
def __init__(
self,
prompt: str = "",
negative_prompt: str = "",
first_frame: Optional[Tensor] = None,
first_frame_url: str = "",
last_frame: Optional[Tensor] = None,
last_frame_url: str = "",
reference_video_url: str = "",
refer_type: str = "feature",
keep_original_sound: str = "no",
element_id: int = 0,
aspect_ratio: str = "16:9",
duration: int = 5,
mode: str = "std",
sound: str = "off",
**kwargs,
):
super().__init__(**kwargs)
self.prompt = normalize_prompt(prompt)
self.negative_prompt = normalize_prompt(negative_prompt)
self.first_frame = first_frame
self.first_frame_url = first_frame_url
self.last_frame = last_frame
self.last_frame_url = last_frame_url
self.reference_video_url = reference_video_url
self.refer_type = refer_type
self.keep_original_sound = keep_original_sound
self.element_id = element_id
self.aspect_ratio = aspect_ratio
self.duration = duration
self.mode = mode
self.sound = sound
def build_input(self) -> Dict[str, Any]:
task_input: Dict[str, Any] = {}
if self.prompt:
task_input["prompt"] = self.prompt
if self.negative_prompt:
task_input["negative_prompt"] = self.negative_prompt
if not task_input.get("prompt"):
raise ValueError(
"Prompt is required for Kling V3 Omni. "
f"Current prompt is empty (received: {self.prompt!r})"
)
return task_input
def build_image_list(self) -> List[Dict[str, str]]:
image_list: List[Dict[str, str]] = []
first_url = resolve_image(self.first_frame, self.first_frame_url, "First frame")
last_url = resolve_image(self.last_frame, self.last_frame_url, "Last frame")
if first_url:
image_list.append({"image_url": first_url, "type": "first_frame"})
if last_url:
image_list.append({"image_url": last_url, "type": "end_frame"})
return image_list
def build_parameters(self) -> Dict[str, Any]:
has_video = bool(self.reference_video_url and self.reference_video_url.strip())
if has_video and self.sound == "on":
raise ValueError("sound must be 'off' when reference_video_url is provided")
params: Dict[str, Any] = {
"duration": self.duration,
"aspect_ratio": self.aspect_ratio,
"mode": self.mode,
"sound": self.sound,
}
image_list = self.build_image_list()
if image_list:
params["image_list"] = image_list
if has_video:
params["video_list"] = [{
"video_url": self.reference_video_url.strip(),
"refer_type": self.refer_type,
"keep_original_sound": self.keep_original_sound,
}]
if self.element_id:
params["element_list"] = [{"element_id": self.element_id}]
return params
def build_payload(self) -> Dict[str, Any]:
return {
"model": MODEL_KLING_V3_OMNI,
"input": self.build_input(),
"parameters": self.build_parameters(),
}
def field_required(self):
return ["prompt"]
def field_order(self):
return [
"prompt",
"negative_prompt",
"first_frame",
"last_frame",
"reference_video_url",
"refer_type",
"keep_original_sound",
"element_id",
"aspect_ratio",
"duration",
"mode",
"sound",
]
+73
View File
@@ -0,0 +1,73 @@
from typing import Optional
from pydantic import Field
from ..utils import BaseRequest
class QwenImage(BaseRequest):
"""
Qwen/Qwen-Image text-to-image via /v1/images/generations.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(..., description="Prompt text")
aspect_ratio: Optional[str] = Field(default="1:1", description="Aspect ratio e.g. 16:9, 1:1")
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images (n)")
seed: Optional[int] = Field(default=-1, description="Random seed (-1 for random)")
steps: Optional[int] = Field(default=20, ge=1, le=50, description="Inference steps")
guidance_scale: Optional[float] = Field(default=2.5, ge=1.0, le=10.0, description="Guidance scale 1~10")
negative_prompt: Optional[str] = Field(default="", description="Negative prompt")
response_format: Optional[str] = Field(default="url", description='"url" or "b64_json"')
def __init__(
self,
prompt: str,
aspect_ratio: Optional[str] = "1:1",
num_images: Optional[int] = 1,
seed: Optional[int] = -1,
steps: Optional[int] = 20,
guidance_scale: Optional[float] = 2.5,
negative_prompt: Optional[str] = "",
response_format: Optional[str] = "url",
**kwargs,
):
super().__init__(**kwargs)
self.prompt = prompt
self.aspect_ratio = aspect_ratio
self.num_images = num_images
self.seed = seed
self.steps = steps
self.guidance_scale = guidance_scale
self.negative_prompt = negative_prompt
self.response_format = response_format
def build_payload(self) -> dict:
payload = {
"model": "Qwen/Qwen-Image",
"prompt": self.prompt,
"aspect_ratio": self.aspect_ratio,
"n": self.num_images,
"seed": self.seed,
"steps": self.steps,
"guidance_scale": self.guidance_scale,
"negative_prompt": self.negative_prompt,
"response_format": self.response_format,
}
return self._remove_empty_fields(payload)
def field_required(self):
return ["prompt"]
def field_order(self):
return [
"model",
"prompt",
"aspect_ratio",
"n",
"seed",
"steps",
"guidance_scale",
"negative_prompt",
"response_format",
]
@@ -0,0 +1,81 @@
from typing import Optional
from pydantic import Field
from ..utils import BaseRequest, image_to_base64
from torch import Tensor
class QwenImageEdit(BaseRequest):
"""
Qwen/Qwen-Image-Edit request model for /v1/images/generations (OpenAI-compatible).
Supports image edit with a prompt and optional controls.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(..., description="Prompt text for the edit")
image: Tensor = Field(..., description="Input image tensor for edit")
negative_prompt: Optional[str] = Field(default="", description="Negative prompt")
num_images: Optional[int] = Field(default=1, ge=1, le=4, description="Number of images to generate (n)")
strength: Optional[float] = Field(default=0.8, ge=0.0, le=1.0, description="Edit strength 0~1")
seed: Optional[int] = Field(default=-1, description="Random seed (-1 for random)")
steps: Optional[int] = Field(default=20, ge=1, le=50, description="Inference steps 1~50")
guidance_scale: Optional[float] = Field(default=2.5, ge=1.0, le=10.0, description="Guidance scale 1~10")
response_format: Optional[str] = Field(default="url", description='"url" or "b64_json"')
def __init__(
self,
prompt: str,
image: Tensor,
negative_prompt: Optional[str] = "",
num_images: Optional[int] = 1,
strength: Optional[float] = 0.8,
seed: Optional[int] = -1,
steps: Optional[int] = 20,
guidance_scale: Optional[float] = 2.5,
response_format: Optional[str] = "url",
**kwargs,
):
super().__init__(**kwargs)
self.prompt = prompt
self.negative_prompt = negative_prompt
self.num_images = num_images
self.strength = strength
self.seed = seed
self.steps = steps
self.guidance_scale = guidance_scale
self.response_format = response_format
self.image = image_to_base64(image)
def build_payload(self) -> dict:
payload = {
"model": "Qwen/Qwen-Image-Edit",
"prompt": self.prompt,
"image": self.image,
"negative_prompt": self.negative_prompt,
"n": self.num_images,
"strength": self.strength,
"seed": self.seed,
"steps": self.steps,
"guidance_scale": self.guidance_scale,
"response_format": self.response_format,
}
return self._remove_empty_fields(payload)
def field_required(self):
return ["prompt", "image"]
def field_order(self):
return [
"model",
"prompt",
"image",
"negative_prompt",
"n",
"strength",
"seed",
"steps",
"guidance_scale",
"response_format",
]
@@ -11,6 +11,7 @@ class Step1xEdit(BaseRequest):
Step1X-Edit transforms your photos with simple instructions into stunning, professional-quality edits—rivaling top proprietary tools.
"""
API_PATH = "/v1/images/generations"
prompt: str = Field(...,
description="The prompt to generate an image from.")
image: Tensor = Field(
+133 -19
View File
@@ -1,5 +1,10 @@
import os
import json
import configparser
import torch
import server
from aiohttp import web
from comfy.comfy_types.node_typing import IO
try:
current_dir = os.path.dirname(os.path.abspath(__file__))
@@ -18,6 +23,61 @@ except Exception as e:
config = None
secrets_path = os.path.join(parent_dir, 'secrets.json')
def load_secrets():
if not os.path.exists(secrets_path):
with open(secrets_path, 'w') as secrets_file:
json.dump({}, secrets_file, indent=2)
with open(secrets_path) as secrets_file:
return json.load(secrets_file)
def save_secrets(secrets):
with open(secrets_path, 'w') as secrets_file:
json.dump(secrets, secrets_file, indent=2)
async def get_modelverse_secrets(_request):
return web.json_response(load_secrets())
async def set_modelverse_secret(request):
try:
data = await request.json()
except Exception:
return web.json_response({"error": "Invalid or empty request body"}, status=400)
key = data.get("key", "").strip()
value = data.get("value", "")
if not key:
return web.json_response({"error": "Key cannot be empty"}, status=400)
secrets = load_secrets()
secrets[key] = value
save_secrets(secrets)
return web.json_response({"ok": True})
async def delete_modelverse_secret(request):
key = request.match_info["key"]
secrets = load_secrets()
if key not in secrets:
return web.json_response({"error": "Key not found"}, status=404)
del secrets[key]
save_secrets(secrets)
return web.json_response({"ok": True})
if not getattr(server.PromptServer.instance, "_modelverse_secrets_registered", False):
server.PromptServer.instance.routes.get("/modelverse-secrets")(get_modelverse_secrets)
server.PromptServer.instance.routes.post("/modelverse-secrets")(set_modelverse_secret)
server.PromptServer.instance.routes.delete("/modelverse-secrets/{key}")(delete_modelverse_secret)
server.PromptServer.instance._modelverse_secrets_registered = True
class ModelverseAPIClient:
"""
Ucloud Modelverse API Client Node
@@ -32,7 +92,11 @@ class ModelverseAPIClient:
def INPUT_TYPES(s):
return {
"required": {
"api_key": ("STRING", {"multiline": False, "default": ""})
"api_key": ("STRING", {
"multiline": False,
"default": "",
"tooltip": "ModelVerse API key. Leave empty to read MODELVERSE_API_KEY from config.ini and avoid storing the key in workflow files."
})
},
}
@@ -71,36 +135,82 @@ class ModelverseAPIClient:
},)
class ModelverseImagePacker:
class ModelverseSecretClient:
"""
Ucloud Modelverse Image Packer
UCloud Modelverse API Client from a locally managed secret.
This node packs multiple images into a list for multi-image editing.
Args:
images1: The first image/image_list to be packed together with.
images2: The second image/image_list to be packed together with. et cetera.
Returns:
image_list: the pack of all input images.
This node stores only the secret name in the workflow. The actual API key is
read from secrets.json at execution time.
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"images1": ("IMAGE,IMAGE_LIST", {"tooltip": "The first image/list to be packed together. Add more if you need."})
"secret": ("STRING", {
"multiline": False,
"default": "",
"tooltip": "Secret name from Modelverse Secrets Manager. The workflow stores this name, not the API key."
})
},
}
RETURN_TYPES = ("MODELVERSE_API_CLIENT",)
RETURN_NAMES = ("client",)
FUNCTION = "create_client"
CATEGORY = "UCLOUD_MODELVERSE"
@classmethod
def IS_CHANGED(cls, secret):
return load_secrets().get(secret, "")
def create_client(self, secret):
secret = secret.strip() if isinstance(secret, str) else secret
if not secret:
raise ValueError("Secret name is required")
secrets = load_secrets()
api_key = secrets.get(secret)
if not api_key:
raise ValueError(f"Secret '{secret}' not found in secrets.json")
return ({
"api_key": api_key
},)
class ModelverseImagePacker:
"""
Ucloud Modelverse Image Packer
This node packs multiple images into a batched IMAGE tensor for multi-image editing.
Args:
images1: The first image to be packed together with.
images2: The second image to be packed together with, et cetera.
Returns:
images: batched IMAGE tensor for Flux Kontext Pro/Max multi-image mode.
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"images1": (IO.IMAGE, {"tooltip": "The first image/list to be packed together. Add more if you need."})
},
"optional": {
"images2": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The second image/list to be packed together."}),
"images3": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The third image/list to be packed together."}),
"images4": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The fourth image/list to be packed together."}),
"images5": ("IMAGE,IMAGE_LIST", {"default": None, "tooltip": "The fifth image/list to be packed together."})
"images2": (IO.IMAGE, {"default": None, "tooltip": "The second image/list to be packed together."}),
"images3": (IO.IMAGE, {"default": None, "tooltip": "The third image/list to be packed together."}),
"images4": (IO.IMAGE, {"default": None, "tooltip": "The fourth image/list to be packed together."}),
"images5": (IO.IMAGE, {"default": None, "tooltip": "The fifth image/list to be packed together."})
}
}
RETURN_TYPES = ("IMAGE_LIST",)
RETURN_NAMES = ("image_list",)
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("images",)
FUNCTION = "pack_images"
@@ -121,14 +231,18 @@ class ModelverseImagePacker:
for i in (images1, images2, images3, images4, images5):
if i is not None:
result.extend(to_list(i))
return (result,)
if not result:
raise ValueError("At least one image is required")
return (torch.cat(result, dim=0),)
NODE_CLASS_MAPPINGS = {
'UCloud ModelVerse Client': ModelverseAPIClient,
'UCloud ModelVerse Secret Client': ModelverseSecretClient,
'ModelVerse Image Packer': ModelverseImagePacker
}
NODE_DISPLAY_NAME_MAPPINGS = {
'UCloud ModelVerse Client': 'Modelverse Client',
'UCloud ModelVerse Secret Client': 'Modelverse Secret Client',
'ModelVerse Image Packer': 'Modelverse Image Packer'
}
+503
View File
@@ -0,0 +1,503 @@
import io
import os
import json
import openai
from openai import OpenAI
import base64
import numpy as np
from PIL import Image
from typing import Optional, List, Dict, Any
from .modelverse_api.client import ModelverseClient
from comfy.comfy_types.node_typing import IO
from server import PromptServer
import folder_paths
# Hardcoded models list
DEFAULT_MODELS = [
'ByteDance/doubao-seed-1.6',
'Qwen/QwQ-32B',
'Qwen/Qwen-Image',
'Qwen/Qwen-Image-Edit',
'Qwen/Qwen3-32B',
'Qwen/Qwen3-Coder',
'doubao-seedance-2-0-260128',
'kling-v3',
'kling-v3-omni',
'happyhorse-1.0-t2v',
'happyhorse-1.0-i2v',
'happyhorse-1.0-r2v',
'veo-3.1-generate-001',
'veo-3.1-fast-generate-001',
'baidu/ernie-4.5-turbo-128k',
'baidu/ernie-4.5-turbo-vl-32k',
'baidu/ernie-x1-turbo-32k',
'black-forest-labs/flux-kontext-max',
'black-forest-labs/flux-kontext-max/multi',
'black-forest-labs/flux-kontext-max/text-to-image',
'black-forest-labs/flux-kontext-pro',
'black-forest-labs/flux-kontext-pro/multi',
'black-forest-labs/flux-kontext-pro/text-to-image',
'black-forest-labs/flux.1-dev',
'claude-4-sonnet',
'deepseek-ai/DeepSeek-R1',
'deepseek-ai/DeepSeek-R1-0528',
'deepseek-ai/DeepSeek-V3-0324',
'deepseek-ai/DeepSeek-V3.1',
'gemini-2.5-flash',
'gemini-2.5-flash-image',
'gemini-2.5-pro',
'gemini-3.1-flash-image',
'gemini-3-pro-image',
'gpt-4.1-mini',
'grok-4',
'moonshotai/Kimi-K2-Instruct',
'openai/gpt-4.1',
'openai/gpt-5',
'openai/gpt-5-mini',
'viduq2',
'viduq2-pro',
'viduq2-turbo',
'viduq2-pro-fast',
'viduq3-pro',
'viduq3-turbo',
'zai-org/glm-5',
'stepfun-ai/step1x-edit'
]
# Default selected model
DEFAULT_MODEL = "zai-org/glm-5"
class ModelverseChat:
"""OpenAI Chat node with support for text, images, and files"""
# Class-level cache for models list
_cached_models = None
_cache_initialized = False
def __init__(self):
pass
@classmethod
def get_models_list(cls):
"""Try to fetch models from ModelVerse API, fallback to default models if failed"""
# Return cached models if already fetched
if cls._cache_initialized:
return cls._cached_models
try:
# Try to create a client and fetch models
# Note: This will work if there's a valid API key available, otherwise fallback
client = OpenAI(base_url="https://api.modelverse.cn/v1",api_key="xxxsu")
models_response = client.models.list()
# Extract model IDs from the response
if hasattr(models_response, 'data') and models_response.data:
model_ids = []
for model in models_response.data:
if hasattr(model, 'id') and model.id:
model_ids.append(model.id)
if model_ids:
print(f"ModelverseChat: Successfully fetched {len(model_ids)} models from API")
cls._cached_models = sorted(model_ids) # Sort for better user experience
cls._cache_initialized = True
return cls._cached_models
# If no models found, fall back to default
print("ModelverseChat: No models found in API response, using default models")
cls._cached_models = DEFAULT_MODELS
cls._cache_initialized = True
return cls._cached_models
except Exception as e:
print(f"ModelverseChat: Failed to fetch models from API ({str(e)}), using default models")
cls._cached_models = DEFAULT_MODELS
cls._cache_initialized = True
return cls._cached_models
@classmethod
def clear_models_cache(cls):
"""Clear the cached models list to force a refresh on next call"""
cls._cached_models = None
cls._cache_initialized = False
print("ModelverseChat: Models cache cleared")
def display_message_on_node(self, message: str, node_id: str) -> None:
"""Display the current response message on the node UI."""
render_spec = {
"node_id": node_id,
"component": "MessageDisplayWidget",
"props": {
"message": message,
},
}
PromptServer.instance.send_sync(
"display_component",
render_spec,
)
@classmethod
def INPUT_TYPES(cls):
# Get models list dynamically
available_models = cls.get_models_list()
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (available_models, {"default": DEFAULT_MODEL if DEFAULT_MODEL in available_models else available_models[0] if available_models else DEFAULT_MODEL}),
"user_prompt": (IO.STRING, {
"multiline": True,
"default": "What can you tell me about this?"
}),
"temperature": (IO.FLOAT, {
"default": 0.7,
"min": 0.0,
"max": 2.0,
"step": 0.1
}),
"max_tokens": (IO.INT, {
"default": 6000,
"min": 1,
"max": 128000
}),
"top_p": (IO.FLOAT, {
"default": 1.0,
"min": 0.0,
"max": 1.0,
"step": 0.01
}),
},
"optional": {
"system_prompt": (IO.STRING, {
"multiline": True,
"default": "You are a helpful assistant."
}),
"image_in": (IO.IMAGE, {}),
"files": ("OPENAI_INPUT_FILES", {
"default": None,
"tooltip": "Optional file(s) to use as context for the model. Accepts inputs from the OpenAI Input Files node."
}),
# "response_format": (["text", "json_object"], {"default": "text"}),
"response_format": (["text"], {"default": "text"}),
"presence_penalty": (IO.FLOAT, {
"default": 0.0,
"min": -2.0,
"max": 2.0,
"step": 0.1
}),
"frequency_penalty": (IO.FLOAT, {
"default": 0.0,
"min": -2.0,
"max": 2.0,
"step": 0.1
}),
},
"hidden": {
"unique_id": "UNIQUE_ID",
},
}
RETURN_TYPES = (IO.STRING,)
RETURN_NAMES = ("response",)
CATEGORY = "UCLOUD_MODELVERSE"
FUNCTION = "chat"
@classmethod
def IS_CHANGED(cls, **kwargs):
"""Tell ComfyUI that this node's inputs may change"""
return "static"
def chat(self,
client: Dict[str, str],
model: str,
user_prompt: str,
temperature: float,
max_tokens: int,
top_p: float,
unique_id: Optional[str] = None,
system_prompt: Optional[str] = "You are a helpful assistant.",
image_in: Optional[Any] = None,
files: Optional[List[Any]] = None,
response_format: str = "text",
presence_penalty: float = 0.0,
frequency_penalty: float = 0.0) -> tuple:
# Create ModelverseClient and get API key
api_key = client.get("api_key")
if not api_key:
raise ValueError("No API key found in the client")
# Create ModelverseClient instance to get the actual API key
modelverse_client = ModelverseClient(api_key)
# Initialize OpenAI client with the API key from ModelverseClient
openai_client = openai.OpenAI(api_key=modelverse_client.api_key,base_url="https://api.modelverse.cn/v1")
# Build messages
messages = []
# Add system message if provided
if system_prompt:
messages.append({
"role": "system",
"content": system_prompt
})
# Build user message content
user_content = []
# Add main prompt
user_content.append({
"type": "text",
"text": user_prompt
})
# Add file content if provided
if files:
print(f"OpenAIChat: Processing {len(files)} files")
for file_info in files:
# Handle the actual file format from OpenAIInputFiles
if hasattr(file_info, 'file_data') and hasattr(file_info, 'filename'):
# Decode base64 content
try:
# Extract base64 data (remove data:text/plain;base64, prefix)
if file_info.file_data.startswith('data:'):
base64_data = file_info.file_data.split(',', 1)[1]
else:
base64_data = file_info.file_data
# Decode base64 to text
content = base64.b64decode(base64_data).decode('utf-8')
file_text = f"\n\nFile: {file_info.filename}\nContent:\n{content}"
user_content.append({
"type": "text",
"text": file_text
})
print(f"OpenAIChat: Added file {file_info.filename} with {len(content)} characters")
except Exception as e:
print(f"OpenAIChat: Error decoding file {file_info.filename}: {str(e)}")
elif isinstance(file_info, dict) and "content" in file_info and "filename" in file_info:
# Handle simple dict format (backup)
file_text = f"\n\nFile: {file_info['filename']}\nContent:\n{file_info['content']}"
user_content.append({
"type": "text",
"text": file_text
})
print(f"OpenAIChat: Added file {file_info['filename']} with {len(file_info['content'])} characters")
else:
print(f"OpenAIChat: Invalid file format: {file_info}")
else:
print("OpenAIChat: No files provided")
# Add image if provided
if image_in is not None:
# Convert tensor to PIL Image
# Handle both single images and batches
if len(image_in.shape) == 4:
# Take first image from batch
image_in = image_in[0]
# Convert from tensor format (H, W, C) to numpy array
image_array = image_in.cpu().numpy()
# Ensure values are in 0-255 range
image_array = np.clip(255. * image_array, 0, 255).astype(np.uint8)
# Create PIL Image
pil_image = Image.fromarray(image_array)
# Convert PIL Image to base64
buffered = io.BytesIO()
pil_image.save(buffered, format="PNG")
img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
# Add image to content
user_content.append({
"type": "image_url",
"image_url": {
"url": f"data:image/png;base64,{img_str}"
}
})
# Add user message with all content
if len(user_content) == 1:
# If only text, use simple format
messages.append({
"role": "user",
"content": user_content[0]["text"]
})
else:
# If multiple content types, use array format
messages.append({
"role": "user",
"content": user_content
})
# Prepare API call parameters
api_params = {
"model": model,
"messages": messages,
"temperature": temperature,
"max_tokens": max_tokens,
"top_p": top_p,
"presence_penalty": presence_penalty,
"frequency_penalty": frequency_penalty,
}
# Add response format if JSON is requested
if response_format == "json_object":
api_params["response_format"] = {"type": "json_object"}
try:
# Make API call to OpenAI
response = openai_client.chat.completions.create(**api_params)
# Extract response content
if response.choices and len(response.choices) > 0:
content = response.choices[0].message.content
if content is None:
raise ValueError("No content in response")
# Display the response message on the node UI
if unique_id:
self.display_message_on_node(content.strip(), unique_id)
# Return the response as a tuple
return (content.strip(),)
else:
raise ValueError("No choices in response")
except Exception as e:
error_msg = f"OpenAI API Error: {str(e)}"
print(error_msg)
return (error_msg,)
class ModelverseInputFiles:
"""
Loads and formats input files for OpenAI API.
"""
@classmethod
def INPUT_TYPES(cls):
"""
For details about the supported file input types, see:
https://platform.openai.com/docs/guides/pdf-files?api-mode=responses
"""
input_dir = folder_paths.get_input_directory()
input_files = [
f
for f in os.scandir(input_dir)
if f.is_file()
and (f.name.endswith(".txt") or f.name.endswith(".pdf") or f.name.endswith(".md"))
and f.stat().st_size < 32 * 1024 * 1024
]
input_files = sorted(input_files, key=lambda x: x.name)
input_files = [f.name for f in input_files]
return {
"required": {},
"optional": {
"file": (
IO.COMBO,
{
"options": input_files,
"default": input_files[0] if input_files else None,
"tooltip": "Input files to include as context for the model. Only accepts text (.txt), markdown (.md) and PDF (.pdf) files for now.",
},
),
"OPENAI_INPUT_FILES": (
"OPENAI_INPUT_FILES",
{
"tooltip": "An optional additional file(s) to batch together with the file loaded from this node. Allows chaining of input files so that a single message can include multiple input files.",
"default": None,
},
),
},
}
DESCRIPTION = "Loads and prepares input files (text, markdown, pdf, etc.) to include as inputs for the OpenAI Chat Node. The files will be read by the OpenAI model when generating a response. 🛈 TIP: Can be chained together with other OpenAI Input File nodes."
RETURN_TYPES = ("OPENAI_INPUT_FILES",)
FUNCTION = "prepare_files"
CATEGORY = "UCLOUD_MODELVERSE"
def read_file_content(self, file_path: str) -> str:
"""Read content from a file."""
try:
with open(file_path, 'r', encoding='utf-8') as f:
return f.read()
except UnicodeDecodeError:
# Try with different encoding if UTF-8 fails
try:
with open(file_path, 'r', encoding='gbk') as f:
return f.read()
except:
return f"Error: Unable to read file {file_path} - encoding issue"
except Exception as e:
return f"Error reading file {file_path}: {str(e)}"
def text_to_data_uri(self, content: str) -> str:
"""Convert text content to data URI."""
encoded_content = base64.b64encode(content.encode('utf-8')).decode('ascii')
return f"data:text/plain;base64,{encoded_content}"
def create_input_file_content(self, file_path: str):
"""Create a file content object compatible with the reference implementation."""
content = self.read_file_content(file_path)
# Create a simple object that mimics the expected structure
class InputFileContent:
def __init__(self, content, filename):
self.file_data = self.text_to_data_uri(content)
self.filename = filename
self.type = "input_file"
self.file_id = None
def text_to_data_uri(self, content: str) -> str:
encoded_content = base64.b64encode(content.encode('utf-8')).decode('ascii')
return f"data:text/plain;base64,{encoded_content}"
return InputFileContent(content, os.path.basename(file_path))
def prepare_files(
self, file: str = None, OPENAI_INPUT_FILES: List[Any] = None
) -> tuple:
"""
Loads and formats input files for OpenAI API.
"""
files = []
# Add current file if provided
if file is not None and file != "":
try:
file_path = folder_paths.get_annotated_filepath(file)
input_file_content = self.create_input_file_content(file_path)
files.append(input_file_content)
print(f"OpenAIInputFiles: Successfully loaded file {file}")
except Exception as e:
print(f"OpenAIInputFiles: Error loading file {file}: {str(e)}")
# Add previous files if provided
if OPENAI_INPUT_FILES is not None:
files.extend(OPENAI_INPUT_FILES)
print(f"OpenAIInputFiles: Added {len(OPENAI_INPUT_FILES)} previous files")
print(f"OpenAIInputFiles: Returning {len(files)} files total")
return (files,)
# Node registration
NODE_CLASS_MAPPINGS = {
"ModelverseChat": ModelverseChat,
"ModelverseInputFiles": ModelverseInputFiles,
# "OpenAICaptionImage": OpenAICaptionImage,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"ModelverseChat": "Modelverse Chat",
"ModelverseInputFiles": "Modelverse Input Files",
# "OpenAICaptionImage": "OpenAI Caption Image",
}
+72
View File
@@ -0,0 +1,72 @@
import os
import re
import requests
import folder_paths
from comfy.comfy_types.node_typing import IO
from comfy_api_nodes.util import download_url_to_video_output
class ModelversePreviewVideo:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"video_url": (IO.STRING, {"forceInput": True}),
"filename_prefix": (IO.STRING, {"default": "Modelverse"}),
"save_output": (IO.BOOLEAN, {"default": True}),
}
}
OUTPUT_NODE = True
FUNCTION = "run"
CATEGORY = "UCLOUD_MODELVERSE"
RETURN_TYPES = (IO.VIDEO,)
RETURN_NAMES = ("video",)
async def run(self, video_url, filename_prefix, save_output):
if type(video_url) == list:
video_url = video_url[0]
# Save to file if requested (do this first before download_url_to_video_output consumes the data)
if save_output:
output_dir = folder_paths.get_output_directory()
(
full_output_folder,
filename,
_,
_,
_,
) = folder_paths.get_save_image_path(filename_prefix, output_dir)
max_counter = 0
matcher = re.compile(f"{re.escape(filename)}_(\\d+)\\D*\\..+", re.IGNORECASE)
for existing_file in os.listdir(full_output_folder):
match = matcher.fullmatch(existing_file)
if match:
file_counter = int(match.group(1))
if file_counter > max_counter:
max_counter = file_counter
counter = max_counter + 1
file = f"{filename}_{counter:05}.mp4"
file_path = os.path.join(full_output_folder, file)
response = requests.get(video_url, timeout=120)
response.raise_for_status()
with open(file_path, "wb") as f:
f.write(response.content)
# Download video and get VIDEO output
video = await download_url_to_video_output(video_url)
return {"ui": {"video_url": [video_url]}, "result": (video,)}
NODE_CLASS_MAPPINGS = {
"Modelverse_PreviewVideo": ModelversePreviewVideo,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse_PreviewVideo": "Modelverse Preview Video",
}
+129
View File
@@ -0,0 +1,129 @@
import base64
import torch
from typing import Optional, List
from comfy.comfy_types.node_typing import IO
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.qwen_image_edit import QwenImageEdit
class QwenImageEditNode:
"""
Qwen/Qwen-Image-Edit image editing via Modelverse /v1/images/generations API.
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "Convert to quick pencil sketch"}),
"image": (IO.IMAGE, {"tooltip": "Input image to edit"}),
"negative_prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "Negative prompt"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
"strength": (IO.FLOAT, {"default": 0.8, "min": 0.0, "max": 1.0, "step": 0.01, "display": "number"}),
"steps": (IO.INT, {"default": 20, "min": 1, "max": 50, "step": 1, "display": "number"}),
"seed": (IO.INT, {"default": -1, "min": -1, "max": 0xffffffffffffffff, "control_after_generate": True}),
"guidance_scale": (IO.FLOAT, {"default": 2.5, "min": 1.0, "max": 10.0, "step": 0.1, "display": "number"}),
"response_format": (["url", "b64_json"], {"default": "url"}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Qwen-Image"
FUNCTION = "execute"
async def execute(
self,
client,
prompt: str,
image,
negative_prompt: str = "",
num_requests: int = 1,
num_images: int = 1,
strength: float = 0.8,
steps: int = 20,
seed: int = -1,
guidance_scale: float = 2.5,
response_format: str = "url",
):
if not prompt:
raise ValueError("Prompt is required")
if image is None:
raise ValueError("Input image is required")
print("INFO:", "Running Qwen/Qwen-Image-Edit.")
mv_client = ModelverseClient(client["api_key"])
tasks = [
mv_client.async_send_request(
QwenImageEdit(
prompt=prompt,
image=image,
negative_prompt=negative_prompt,
num_images=num_images,
strength=strength,
seed=seed + i,
steps=steps,
guidance_scale=guidance_scale,
response_format=response_format,
)
)
for i in range(num_requests)
]
results = await mv_client.run_tasks(tasks) # list of data lists
output_images_list: List[torch.Tensor] = []
for data_list in results:
if not data_list:
print("WARN:", "No output in current request. Skipping...")
continue
if response_format == "url":
output_images = imageurl2tensor(data_list)
else:
# b64_json path
images = []
for item in data_list:
b64v = item.get("b64_json") or item.get("b64")
if not b64v:
continue
if isinstance(b64v, str) and b64v.startswith("data:"):
try:
b64v = b64v.split(",", 1)[1]
except Exception:
pass
try:
img_bytes = base64.b64decode(b64v)
pil_img = decode_image(img_bytes)
images.append(pil_img)
except Exception:
continue
if not images:
print("WARN:", "No decodable base64 image found.")
continue
output_images = images2tensor(images)
output_images_list.append(output_images)
if not output_images_list:
return (torch.zeros((1, 3, 1, 1)),)
return (torch.cat(output_images_list, dim=0),)
NODE_CLASS_MAPPINGS = {
"Modelverse QwenImageEditNode": QwenImageEditNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse QwenImageEditNode": "Modelverse Qwen Image Edit",
}
+122
View File
@@ -0,0 +1,122 @@
import base64
import torch
from typing import List
from comfy.comfy_types.node_typing import IO
from .modelverse_api.utils import imageurl2tensor, decode_image, images2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.qwen_image import QwenImage
class QwenImageT2INode:
"""
Qwen/Qwen-Image text-to-image via Modelverse /v1/images/generations API.
"""
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower"}),
"aspect_ratio": (["21:9", "16:9", "4:3", "3:2", "1:1", "2:3", "3:4", "9:16", "9:21"], {"default": "1:1"}),
"num_requests": (IO.INT, {"default": 1, "min": 1, "max": 10, "step": 1, "display": "number"}),
"num_images": (IO.INT, {"default": 1, "min": 1, "max": 4, "step": 1, "display": "number"}),
"steps": (IO.INT, {"default": 20, "min": 1, "max": 50, "step": 1, "display": "number"}),
"seed": (IO.INT, {"default": -1, "min": -1, "max": 0xffffffffffffffff, "control_after_generate": True}),
"guidance_scale": (IO.FLOAT, {"default": 2.5, "min": 1.0, "max": 10.0, "step": 0.1, "display": "number"}),
"response_format": (["url", "b64_json"], {"default": "url"}),
},
"optional": {
"negative_prompt": (IO.STRING, {"multiline": True, "default": ""}),
},
}
RETURN_TYPES = (IO.IMAGE,)
RETURN_NAMES = ("image",)
CATEGORY = "UCLOUD_MODELVERSE/Qwen-Image"
FUNCTION = "execute"
async def execute(
self,
client,
prompt: str,
aspect_ratio: str = "1:1",
num_requests: int = 1,
num_images: int = 1,
steps: int = 20,
seed: int = -1,
guidance_scale: float = 2.5,
response_format: str = "url",
negative_prompt: str = "",
):
if not prompt:
raise ValueError("Prompt is required")
mv_client = ModelverseClient(client["api_key"])
tasks = [
mv_client.async_send_request(
QwenImage(
prompt=prompt,
aspect_ratio=aspect_ratio,
num_images=num_images,
seed=seed + i,
steps=steps,
guidance_scale=guidance_scale,
negative_prompt=negative_prompt,
response_format=response_format,
)
)
for i in range(num_requests)
]
results = await mv_client.run_tasks(tasks)
output_images_list: List[torch.Tensor] = []
for data_list in results:
if not data_list:
print("WARN:", "No output in current request. Skipping...")
continue
if response_format == "url":
output_images = imageurl2tensor(data_list)
else:
images = []
for item in data_list:
b64v = item.get("b64_json") or item.get("b64")
if not b64v:
continue
if isinstance(b64v, str) and b64v.startswith("data:"):
try:
b64v = b64v.split(",", 1)[1]
except Exception:
pass
try:
img_bytes = base64.b64decode(b64v)
pil_img = decode_image(img_bytes)
images.append(pil_img)
except Exception:
continue
if not images:
print("WARN:", "No decodable base64 image found.")
continue
output_images = images2tensor(images)
output_images_list.append(output_images)
if not output_images_list:
return (torch.zeros((1, 3, 1, 1)),)
return (torch.cat(output_images_list, dim=0),)
NODE_CLASS_MAPPINGS = {
"Modelverse QwenImageT2INode": QwenImageT2INode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse QwenImageT2INode": "Modelverse Qwen Image",
}
+122
View File
@@ -0,0 +1,122 @@
"""
OpenAI Sora2 Img2Video - 图生视频模型
Models: openai/sora-2/image-to-video, openai/sora-2/image-to-video-pro
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODELS = ["openai/sora-2/image-to-video", "openai/sora-2/image-to-video-pro"]
RESOLUTIONS_PRO = ["720p", "1080p"]
DURATIONS = [4, 8, 12]
class SoraImg2VideoNode:
"""
OpenAI Sora2 Img2Video - 图生视频
Models: image-to-video (普通版), image-to-video-pro (Pro版,支持1080p)
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "openai/sora-2/image-to-video", "tooltip": "image-to-video: 普通版, image-to-video-pro: Pro版(支持1080p)"}),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL (与image二选一)"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "提示词,用于指导视频生成"}),
"resolution": (RESOLUTIONS_PRO, {"default": "720p", "tooltip": "分辨率 (Pro版支持1080p)"}),
"duration": (DURATIONS, {"default": 4, "tooltip": "视频时长(秒): 4, 8, 12"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Sora"
def generate(self, client, model, first_frame_image=None, first_frame_url="",
prompt="", resolution="720p", duration=4):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
# Validate resolution for non-pro model
if model == "openai/sora-2/image-to-video" and resolution == "1080p":
raise ValueError("普通版不支持 1080p,请使用 Pro 版")
mv_client = ModelverseClient(api_key)
# Validate first frame input
has_url = first_frame_url and first_frame_url.strip()
has_image = first_frame_image is not None
if has_url and has_image:
raise ValueError("请提供 first_frame_image 或 first_frame_url,不能同时提供")
if not has_url and not has_image:
raise ValueError("必须提供 first_frame_image 或 first_frame_url")
task_input = {}
if has_url:
task_input["first_frame_url"] = first_frame_url.strip()
else:
task_input["first_frame_url"] = image_to_base64(first_frame_image)
if prompt and prompt.strip():
task_input["prompt"] = prompt.strip()
parameters = {
"duration": duration,
}
# Pro版支持resolution参数
if model == "openai/sora-2/image-to-video-pro":
parameters["resolution"] = resolution
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Sora I2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Sora_Img2Video": SoraImg2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Sora_Img2Video": "Sora Img2Video",
}
+99
View File
@@ -0,0 +1,99 @@
"""
OpenAI Sora2 Text2Video - 文生视频模型
Models: openai/sora-2/text-to-video, openai/sora-2/text-to-video-pro
"""
import time
from .modelverse_api.client import ModelverseClient
from comfy.comfy_types.node_typing import IO
MODELS = ["openai/sora-2/text-to-video", "openai/sora-2/text-to-video-pro"]
SIZES = ["720x1280", "1280x720"]
SIZES_PRO = ["720x1280", "1280x720", "1024x1792", "1792x1024"]
DURATIONS = [4, 8, 12]
class SoraText2VideoNode:
"""
OpenAI Sora2 Text2Video - 文生视频
Models: text-to-video (普通版), text-to-video-pro (Pro版,支持更多分辨率)
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "openai/sora-2/text-to-video", "tooltip": "text-to-video: 普通版, text-to-video-pro: Pro版"}),
"prompt": (IO.STRING, {"multiline": True, "default": "A beautiful girl is dancing", "tooltip": "提示词,用于指导视频生成"}),
},
"optional": {
"size": (SIZES_PRO, {"default": "720x1280", "tooltip": "视频尺寸 (Pro版支持更多选项)"}),
"duration": (DURATIONS, {"default": 4, "tooltip": "视频时长(秒): 4, 8, 12"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Sora"
def generate(self, client, model, prompt, size="720x1280", duration=4):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
# Validate size for non-pro model
if model == "openai/sora-2/text-to-video" and size not in SIZES:
raise ValueError(f"普通版仅支持 {SIZES},请使用 Pro 版获取更多分辨率选项")
mv_client = ModelverseClient(api_key)
task_input = {"prompt": prompt}
parameters = {
"size": size,
"duration": duration,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Sora T2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Sora_Text2Video": SoraText2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Sora_Text2Video": "Sora Text2Video",
}
+9 -9
View File
@@ -2,7 +2,7 @@ from .modelverse_api.utils import imageurl2tensor
from .modelverse_api.client import ModelverseClient
from .modelverse_api.requests.step1x_edit import Step1xEdit
import torch
import asyncio
from comfy.comfy_types.node_typing import IO
class Step1xEditNode:
@@ -17,10 +17,10 @@ class Step1xEditNode:
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "The prompt to guide the image edit."}),
"negative_prompt": ("STRING", {"multiline": True, "default": "", "tooltip": "The negative prompt to use."}),
"image": ("IMAGE", {"default": None, "tooltip": "The image to be edited."}),
"num_requests": ("INT", {
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "The prompt to guide the image edit."}),
"negative_prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "The negative prompt to use."}),
"image": (IO.IMAGE, {"default": None, "tooltip": "The image to be edited."}),
"num_requests": (IO.INT, {
"default": 1,
"min": 1,
"max": 10,
@@ -28,14 +28,14 @@ class Step1xEditNode:
"display": "number",
"tooltip": "Number of request to make (1 to 10)"
}),
"seed": ("INT", {
"seed": (IO.INT, {
"default": -1,
"min": -1,
"max": 0xffffffffffffffff,
"control_after_generate": True,
"tooltip": "Random seed for reproducible results. -1 for random seed"
}),
"num_inference_steps": ("INT", {
"num_inference_steps": (IO.INT, {
"default": 30,
"min": 1,
"max": 50,
@@ -43,7 +43,7 @@ class Step1xEditNode:
"display": "number",
"tooltip": "Number of inference steps (1 to 50)"
}),
"guidance_scale": ("FLOAT", {
"guidance_scale": (IO.FLOAT, {
"default": 4.0,
"min": 0.0,
"max": 20.0,
@@ -54,7 +54,7 @@ class Step1xEditNode:
},
}
RETURN_TYPES = ("IMAGE", )
RETURN_TYPES = (IO.IMAGE, )
RETURN_NAMES = ("image", )
CATEGORY = "UCLOUD_MODELVERSE"
+211
View File
@@ -0,0 +1,211 @@
"""
Google Veo 3.1 video generation
Models: veo-3.1-generate-001, veo-3.1-fast-generate-001
"""
import base64
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import decode_image, encode_image, fetch_image, tensor2images
from comfy.comfy_types.node_typing import IO
MODELS = ["veo-3.1-generate-001", "veo-3.1-fast-generate-001"]
ASPECT_RATIOS = ["16:9", "9:16"]
RESOLUTIONS = ["720p", "1080p"]
DURATIONS = [4, 6, 8]
PERSON_GENERATIONS = ["dont_allow", "allow_adult"]
def _bytes_to_veo_image(data_bytes, fmt):
mime = "image/png" if fmt == "PNG" else "image/jpeg"
return {
"bytesBase64Encoded": base64.b64encode(data_bytes).decode("utf-8"),
"mimeType": mime,
}
def _tensor_to_veo_image(tensor):
data_bytes, fmt = encode_image(tensor2images(tensor)[0])
return _bytes_to_veo_image(data_bytes, fmt)
def _url_to_veo_image(url, label):
url = url.strip()
if url.startswith("data:image/") and "," in url:
header, data = url.split(",", 1)
mime = header.split(";")[0].replace("data:", "")
return {"bytesBase64Encoded": data, "mimeType": mime}
if url.startswith(("http://", "https://")):
image_data = fetch_image(url)
img = decode_image(image_data)
data_bytes, fmt = encode_image(img)
return _bytes_to_veo_image(data_bytes, fmt)
raise ValueError(f"{label}: URL must be http(s) or a data:image/...;base64,... value")
def _resolve_veo_image(image, url, label):
has_url = url and url.strip()
has_image = image is not None
if has_url and has_image:
raise ValueError(f"{label}: provide either image or url, not both")
if has_url:
return _url_to_veo_image(url, label)
if has_image:
return _tensor_to_veo_image(image)
return None
class Veo31VideoNode:
"""
Veo 3.1 video generation.
Supports text-to-video, image-to-video, and first/last-frame video.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {
"default": "veo-3.1-generate-001",
"tooltip": "veo-3.1-generate-001: standard quality, veo-3.1-fast-generate-001: faster generation",
}),
"prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Text prompt describing the desired video",
}),
"generate_audio": (IO.BOOLEAN, {
"default": True,
"tooltip": "Whether to generate synchronized audio (required by Veo API)",
}),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "First frame image for image-to-video"}),
"first_frame_url": (IO.STRING, {
"default": "",
"tooltip": "First frame image URL (use either this OR first_frame_image)",
}),
"last_frame_image": (IO.IMAGE, {"tooltip": "Last frame image for start-end video"}),
"last_frame_url": (IO.STRING, {
"default": "",
"tooltip": "Last frame image URL (use either this OR last_frame_image)",
}),
"negative_prompt": (IO.STRING, {
"multiline": True,
"default": "",
"tooltip": "Negative prompt for unwanted elements",
}),
"aspect_ratio": (ASPECT_RATIOS, {
"default": "16:9",
"tooltip": "Output aspect ratio",
}),
"resolution": (RESOLUTIONS, {
"default": "720p",
"tooltip": "Output resolution",
}),
"duration": (DURATIONS, {
"default": 8,
"tooltip": "Video duration in seconds: 4, 6, or 8",
}),
"seed": (IO.INT, {
"default": 0, "min": 0, "max": 4294967295,
"tooltip": "Random seed (0 to skip)",
}),
"person_generation": (PERSON_GENERATIONS, {
"default": "allow_adult",
"tooltip": "Safety setting for person/face generation",
}),
},
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Veo"
def generate(
self,
client,
model,
prompt,
generate_audio=True,
first_frame_image=None,
first_frame_url="",
last_frame_image=None,
last_frame_url="",
negative_prompt="",
aspect_ratio="16:9",
resolution="720p",
duration=8,
seed=0,
person_generation="allow_adult",
):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
if not prompt or not prompt.strip():
raise ValueError("prompt is required for Veo 3.1")
first_image = _resolve_veo_image(first_frame_image, first_frame_url, "First frame")
last_image = _resolve_veo_image(last_frame_image, last_frame_url, "Last frame")
if last_image and not first_image:
raise ValueError("First frame is required when last frame is provided")
mv_client = ModelverseClient(api_key)
task_input = {"prompt": prompt.strip()}
if negative_prompt and negative_prompt.strip():
task_input["negative_prompt"] = negative_prompt.strip()
if first_image:
task_input["image"] = first_image
if last_image:
task_input["last_frame"] = last_image
parameters = {
"generate_audio": generate_audio,
"aspect_ratio": aspect_ratio,
"resolution": resolution,
"duration": duration,
"person_generation": person_generation,
}
if seed > 0:
parameters["seed"] = seed
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Veo 3.1 task submitted: model={model}, task_id={task_id}")
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
if task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
if task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i + 1}/{max_retries})")
time.sleep(5)
continue
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Veo_3_1_Video": Veo31VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Veo_3_1_Video": "Modelverse Veo 3.1 Video",
}
+120
View File
@@ -0,0 +1,120 @@
"""
Vidu Extend - 视频延长模型
Models: viduq2-pro, viduq2-turbo
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODELS = ["viduq2-pro", "viduq2-turbo"]
RESOLUTIONS = ["540p", "720p", "1080p"]
class ViduExtendNode:
"""
Vidu Extend - 视频延长
Models: viduq2-pro, viduq2-turbo
输入视频时长需4秒-1分钟,延长1-7秒
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "viduq2-turbo", "tooltip": "viduq2-pro: 效果好, viduq2-turbo: 生成快"}),
"video_url": (IO.STRING, {"default": "", "tooltip": "需要延长的视频URL (4秒-1分钟)"}),
"duration": (IO.INT, {"default": 5, "min": 1, "max": 7, "step": 1, "tooltip": "延长时长(秒), 1-7"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
},
"optional": {
"last_frame_image": (IO.IMAGE, {"tooltip": "延长到尾帧的参考图像"}),
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "延长到尾帧的参考图像URL"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "延长提示词,控制延长内容"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
def generate(self, client, model, video_url, duration, resolution,
last_frame_image=None, last_frame_url="", prompt=""):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
if not video_url or not video_url.strip():
raise ValueError("必须提供 video_url")
mv_client = ModelverseClient(api_key)
task_input = {
"video_url": video_url.strip(),
}
# Optional last frame
has_last_url = last_frame_url and last_frame_url.strip()
has_last_image = last_frame_image is not None
if has_last_url and has_last_image:
raise ValueError("尾帧:请提供 image 或 url,不能同时提供")
if has_last_url:
task_input["last_frame_url"] = last_frame_url.strip()
elif has_last_image:
task_input["last_frame_url"] = image_to_base64(last_frame_image)
if prompt and prompt.strip():
task_input["prompt"] = prompt.strip()
parameters = {
"vidu_type": "extend",
"duration": duration,
"resolution": resolution,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Vidu Extend task submitted: {task_id}")
# Poll for result
video_url_result = self._poll_task(mv_client, task_id)
return (video_url_result, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Vidu_Extend": ViduExtendNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Vidu_Extend": "Vidu Extend",
}
+124
View File
@@ -0,0 +1,124 @@
"""
Vidu Img2Video - 图生视频模型
Models: viduq3-pro, viduq3-turbo, viduq2-pro, viduq2-turbo, viduq2-pro-fast
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2-pro", "viduq2-turbo", "viduq2-pro-fast"]
RESOLUTIONS = ["540p", "720p", "1080p"]
MOVEMENT_AMPLITUDES = ["auto", "small", "medium", "large"]
class ViduImg2VideoNode:
"""
Vidu Img2Video - 图生视频
Models: viduq3-pro/viduq3-turbo (1-16s), viduq2-pro/viduq2-turbo/viduq2-pro-fast (1-10s)
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro/turbo: 支持1-16秒; viduq2-pro/turbo/pro-fast: 支持1-10秒"}),
"duration": (IO.INT, {"default": 5, "min": 1, "max": 16, "step": 1, "tooltip": "视频时长(秒),viduq3系列支持1-16秒,viduq2系列支持1-10秒"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
"movement_amplitude": (MOVEMENT_AMPLITUDES, {"default": "auto", "tooltip": "运动幅度"}),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL (与image二选一)"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "文本提示词,最长2000字符"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
def generate(self, client, model, duration, resolution, movement_amplitude,
first_frame_image=None, first_frame_url="", prompt="", seed=0, bgm=False):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
mv_client = ModelverseClient(api_key)
# Validate first frame input
has_url = first_frame_url and first_frame_url.strip()
has_image = first_frame_image is not None
if has_url and has_image:
raise ValueError("请提供 first_frame_image 或 first_frame_url,不能同时提供")
if not has_url and not has_image:
raise ValueError("必须提供 first_frame_image 或 first_frame_url")
if duration > 10 and not model.startswith("viduq3-"):
raise ValueError("只有 viduq3 系列模型支持超过10秒的视频时长")
task_input = {}
if has_url:
task_input["first_frame_url"] = first_frame_url.strip()
else:
task_input["first_frame_url"] = image_to_base64(first_frame_image)
if prompt and prompt.strip():
task_input["prompt"] = prompt.strip()
parameters = {
"vidu_type": "img2video",
"duration": duration,
"resolution": resolution,
"movement_amplitude": movement_amplitude,
"seed": seed,
"bgm": bgm,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Vidu I2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Vidu_Img2Video": ViduImg2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Vidu_Img2Video": "Vidu Img2Video",
}
+138
View File
@@ -0,0 +1,138 @@
"""
Vidu Reference2Video - 参考生视频模型
Models: viduq3-turbo, viduq2
支持1-7张参考图片,生成具备主体一致的视频
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODELS = ["viduq3-turbo", "viduq2"]
ASPECT_RATIOS = ["16:9", "9:16", "3:4", "4:3", "1:1"]
RESOLUTIONS = ["540p", "720p", "1080p"]
class ViduReference2VideoNode:
"""
Vidu Reference2Video - 参考生视频
Models: viduq3-turbo, viduq2
支持1-7张参考图片,生成具备主体一致的视频
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "viduq3-turbo", "tooltip": "viduq3-turbo: 生成快, viduq2: 旧版模型"}),
"prompt": (IO.STRING, {"multiline": True, "default": "make it dance", "tooltip": "文本提示词,最长2000字符"}),
"duration": (IO.INT, {"default": 5, "min": 1, "max": 10, "step": 1, "tooltip": "视频时长(秒)"}),
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "长宽比"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
},
"optional": {
"image1": (IO.IMAGE, {"tooltip": "参考图片1"}),
"image2": (IO.IMAGE, {"tooltip": "参考图片2"}),
"image3": (IO.IMAGE, {"tooltip": "参考图片3"}),
"image4": (IO.IMAGE, {"tooltip": "参考图片4"}),
"image5": (IO.IMAGE, {"tooltip": "参考图片5"}),
"image6": (IO.IMAGE, {"tooltip": "参考图片6"}),
"image7": (IO.IMAGE, {"tooltip": "参考图片7"}),
"image_urls": (IO.STRING, {"default": "", "multiline": True, "tooltip": "参考图片URL列表,每行一个"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
def generate(self, client, model, prompt, duration, aspect_ratio, resolution,
image1=None, image2=None, image3=None, image4=None,
image5=None, image6=None, image7=None,
image_urls="", seed=0, bgm=False):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
mv_client = ModelverseClient(api_key)
# Collect images
images = []
for img in [image1, image2, image3, image4, image5, image6, image7]:
if img is not None:
images.append(image_to_base64(img))
# Add URL images
if image_urls and image_urls.strip():
for url in image_urls.strip().split('\n'):
url = url.strip()
if url:
images.append(url)
if not images:
raise ValueError("至少需要提供1张参考图片")
if len(images) > 7:
print(f"Warning: 最多支持7张参考图片,当前{len(images)}张,将只使用前7张")
images = images[:7]
task_input = {
"images": images,
"prompt": prompt,
}
parameters = {
"vidu_type": "reference2video",
"duration": duration,
"aspect_ratio": aspect_ratio,
"resolution": resolution,
"seed": seed,
"bgm": bgm,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Vidu Ref2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Vidu_Reference2Video": ViduReference2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Vidu_Reference2Video": "Vidu Reference2Video",
}
+144
View File
@@ -0,0 +1,144 @@
"""
Vidu StartEnd2Video - 首尾帧生视频模型
Models: viduq3-pro, viduq3-turbo, viduq2-pro-fast, viduq2-pro, viduq2-turbo
"""
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2-pro-fast", "viduq2-pro", "viduq2-turbo"]
RESOLUTIONS = ["540p", "720p", "1080p"]
MOVEMENT_AMPLITUDES = ["auto", "small", "medium", "large"]
class ViduStartEnd2VideoNode:
"""
Vidu StartEnd2Video - 首尾帧生视频
Models: viduq3-pro/viduq3-turbo (1-16s), viduq2-pro-fast/viduq2-pro/viduq2-turbo (1-8s)
注意:首尾帧分辨率需相近 (0.8~1.25)
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro/turbo: 支持1-16秒; viduq2-pro-fast/pro/turbo: 支持1-8秒"}),
"duration": (IO.INT, {"default": 5, "min": 1, "max": 16, "step": 1, "tooltip": "视频时长(秒),viduq3系列支持1-16秒,viduq2系列支持1-8秒"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
"movement_amplitude": (MOVEMENT_AMPLITUDES, {"default": "auto", "tooltip": "运动幅度"}),
},
"optional": {
"first_frame_image": (IO.IMAGE, {"tooltip": "首帧图片"}),
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "首帧图片URL"}),
"last_frame_image": (IO.IMAGE, {"tooltip": "尾帧图片"}),
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "尾帧图片URL"}),
"prompt": (IO.STRING, {"multiline": True, "default": "", "tooltip": "文本提示词,最长2000字符"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子"}),
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
def generate(self, client, model, duration, resolution, movement_amplitude,
first_frame_image=None, first_frame_url="",
last_frame_image=None, last_frame_url="",
prompt="", seed=0, bgm=False):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
mv_client = ModelverseClient(api_key)
# Validate first frame
has_first_url = first_frame_url and first_frame_url.strip()
has_first_image = first_frame_image is not None
if has_first_url and has_first_image:
raise ValueError("首帧:请提供 image 或 url,不能同时提供")
if not has_first_url and not has_first_image:
raise ValueError("必须提供首帧图片")
# Validate last frame
has_last_url = last_frame_url and last_frame_url.strip()
has_last_image = last_frame_image is not None
if has_last_url and has_last_image:
raise ValueError("尾帧:请提供 image 或 url,不能同时提供")
if not has_last_url and not has_last_image:
raise ValueError("必须提供尾帧图片")
if duration > 8 and not model.startswith("viduq3-"):
raise ValueError("只有 viduq3 系列模型支持超过8秒的视频时长")
task_input = {}
# First frame
if has_first_url:
task_input["first_frame_url"] = first_frame_url.strip()
else:
task_input["first_frame_url"] = image_to_base64(first_frame_image)
# Last frame
if has_last_url:
task_input["last_frame_url"] = last_frame_url.strip()
else:
task_input["last_frame_url"] = image_to_base64(last_frame_image)
if prompt and prompt.strip():
task_input["prompt"] = prompt.strip()
parameters = {
"vidu_type": "start-end2video",
"duration": duration,
"resolution": resolution,
"movement_amplitude": movement_amplitude,
"seed": seed,
"bgm": bgm,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Vidu StartEnd2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Vidu_StartEnd2Video": ViduStartEnd2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Vidu_StartEnd2Video": "Vidu StartEnd2Video",
}
+103
View File
@@ -0,0 +1,103 @@
"""
Vidu Text2Video - 文生视频模型
Models: viduq3-pro, viduq3-turbo, viduq2
"""
import time
from .modelverse_api.client import ModelverseClient
from comfy.comfy_types.node_typing import IO
MODELS = ["viduq3-pro", "viduq3-turbo", "viduq2"]
ASPECT_RATIOS = ["16:9", "9:16", "3:4", "4:3", "1:1"]
RESOLUTIONS = ["540p", "720p", "1080p"]
class ViduText2VideoNode:
"""
Vidu Text2Video - 文生视频
Models: viduq3-pro, viduq3-turbo, viduq2
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"model": (MODELS, {"default": "viduq3-pro", "tooltip": "viduq3-pro: 效果好细节丰富, viduq3-turbo: 生成快, viduq2: 旧版模型"}),
"prompt": (IO.STRING, {"multiline": True, "default": "A beautiful sunset over the ocean", "tooltip": "文本提示词,最长2000字符"}),
"duration": (IO.INT, {"default": 5, "min": 1, "max": 10, "step": 1, "tooltip": "视频时长(秒)"}),
"aspect_ratio": (ASPECT_RATIOS, {"default": "16:9", "tooltip": "长宽比"}),
"resolution": (RESOLUTIONS, {"default": "720p", "tooltip": "分辨率"}),
},
"optional": {
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "随机种子,0表示随机"}),
"guidance_scale": (IO.FLOAT, {"default": 7.5, "min": 1.0, "max": 20.0, "step": 0.5, "tooltip": "引导系数"}),
"bgm": (IO.BOOLEAN, {"default": False, "tooltip": "是否添加背景音乐"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate"
CATEGORY = "UCLOUD_MODELVERSE/Vidu"
def generate(self, client, model, prompt, duration, aspect_ratio, resolution, seed=0, guidance_scale=7.5, bgm=False):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set")
mv_client = ModelverseClient(api_key)
task_input = {"prompt": prompt}
parameters = {
"vidu_type": "text2video",
"duration": duration,
"aspect_ratio": aspect_ratio,
"resolution": resolution,
"seed": seed,
"guidance_scale": guidance_scale,
"bgm": bgm,
}
# Submit task
submit_res = mv_client.submit_task(model, task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res}")
print(f"Vidu T2V task submitted: {task_id}")
# Poll for result
video_url = self._poll_task(mv_client, task_id)
return (video_url, task_id)
def _poll_task(self, mv_client, task_id, max_retries=180):
for i in range(max_retries):
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls:
return urls[0]
raise Exception("Task succeeded but no video URL returned")
elif task_status == "Failure":
error = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id}: {task_status} ({i+1}/{max_retries})")
time.sleep(5)
else:
raise Exception(f"Unknown status: {task_status}")
raise Exception("Task timed out")
NODE_CLASS_MAPPINGS = {
"Vidu_Text2Video": ViduText2VideoNode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Vidu_Text2Video": "Vidu Text2Video",
}
+147
View File
@@ -0,0 +1,147 @@
import time
from .modelverse_api.client import ModelverseClient
from .modelverse_api.utils import image_to_base64
from comfy.comfy_types.node_typing import IO
class Modelverse_WanAII2V:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "Convert to video","tooltip": "Text prompt to guide video generation"}),
},
"optional": {
"first_frame_image": (IO.IMAGE,),
"first_frame_url": (IO.STRING, {"default": "", "tooltip": "First frame image URL (use either this OR first_frame_image, not both)"}),
"last_frame_image": (IO.IMAGE,),
"last_frame_url": (IO.STRING, {"default": "", "tooltip": "Optional: URL for the last frame of the video"}),
"negative_prompt": (IO.STRING, {"multiline": True, "default": "low quality, blurry","tooltip": "Negative prompt to avoid unwanted content"}),
"resolution": (["720P", "480P"], {"default": "720P", "tooltip": "Output video resolution"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647, "tooltip": "Random seed for reproducible results"}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate_video"
CATEGORY = "UCLOUD_MODELVERSE/Wan"
def generate_video(self, client, prompt, first_frame_image=None, first_frame_url="", last_frame_image=None, last_frame_url="", negative_prompt="", resolution="720P", seed=0):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set in the client")
mv_client = ModelverseClient(api_key)
# Prepare the input data
task_input = {"prompt": prompt}
# Validate first frame input - must provide either image or URL, but not both
has_url = first_frame_url and first_frame_url.strip()
has_image = first_frame_image is not None
if has_url and has_image:
raise ValueError("Please provide either first_frame_image OR first_frame_url, not both")
elif not has_url and not has_image:
raise ValueError("Must provide either first_frame_image or first_frame_url")
# Handle first frame
if has_url:
task_input["first_frame_url"] = first_frame_url.strip()
print(f"Using first frame URL: {first_frame_url}")
else:
# Convert IMAGE tensor to base64
first_frame_base64 = image_to_base64(first_frame_image)
if not first_frame_base64:
raise ValueError("Failed to convert first frame image to base64")
task_input["first_frame_url"] = first_frame_base64
print("Using first frame from IMAGE input (converted to base64)")
# Handle last frame (optional)
if last_frame_url and last_frame_url.strip():
task_input["last_frame_url"] = last_frame_url.strip()
print(f"Using last frame URL: {last_frame_url}")
elif last_frame_image is not None:
# Convert IMAGE tensor to base64
last_frame_base64 = image_to_base64(last_frame_image)
if last_frame_base64:
task_input["last_frame_url"] = last_frame_base64
print("Using last frame from IMAGE input (converted to base64)")
# Add negative prompt if provided
if negative_prompt and negative_prompt.strip():
task_input["negative_prompt"] = negative_prompt.strip()
# Set parameters
parameters = {
"resolution": resolution,
"duration": 5, # Fixed as per documentation
"seed": seed,
}
print(f"Submitting I2V task with model: Wan-AI/Wan2.2-I2V")
print(f"Parameters: resolution={resolution}, seed={seed}")
# 1. Submit the task
submit_res = mv_client.submit_task("Wan-AI/Wan2.2-I2V", task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res.get('request_id')}")
print(f"Task submitted successfully with ID: {task_id}")
# 2. Poll for the result
video_url = ""
max_retries = 120 # Maximum 10 minutes (120 * 5 seconds)
retry_count = 0
while True:
try:
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
urls = status_res.get("output", {}).get("urls", [])
if urls and len(urls) > 0:
video_url = urls[0]
print(f"Task completed successfully! Video URL: {video_url}")
break
else:
raise Exception("Task succeeded but no video URL was returned.")
elif task_status == "Failure":
error_message = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error_message}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id} is {task_status}, waiting... ({retry_count + 1}/{max_retries})")
time.sleep(5) # Wait for 5 seconds before polling again
retry_count += 1
else:
raise Exception(f"Unknown task status: {task_status}")
except Exception as e:
if "Task failed" in str(e) or "Unknown task status" in str(e):
raise e
print(f"Error checking task status: {e}, retrying...")
retry_count += 1
time.sleep(5)
if not video_url:
raise Exception(f"Task timed out after {max_retries * 5} seconds")
return (video_url, task_id)
NODE_CLASS_MAPPINGS = {
"Modelverse_WanAII2V": Modelverse_WanAII2V
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse_WanAII2V": "Modelverse Wan-AI I2V"
}
+82
View File
@@ -0,0 +1,82 @@
import time
from .modelverse_api.client import ModelverseClient
from comfy.comfy_types.node_typing import IO
class Modelverse_WanAIT2V:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"client": ("MODELVERSE_API_CLIENT",),
"prompt": (IO.STRING, {"multiline": True, "default": "a beautiful flower","tooltip": "Text prompt of the image to generate"}),
"negative_prompt": (IO.STRING, {"multiline": True, "default": "low quality","tooltip": "Negative prompt of the image to generate"}),
"resolution": (["720P", "480P"], {"default": "720P"}),
"size": (["1280x720", "720x1280", "832x480", "480x832"], {"default": "1280x720"}),
"seed": (IO.INT, {"default": 0, "min": 0, "max": 2147483647}),
}
}
RETURN_TYPES = (IO.STRING, IO.STRING)
RETURN_NAMES = ("url", "task_id")
FUNCTION = "generate_video"
CATEGORY = "UCLOUD_MODELVERSE/Wan"
def generate_video(self, client, prompt, negative_prompt, resolution, size, seed):
api_key = client.get("api_key")
if not api_key:
raise ValueError("API key is not set in the client")
mv_client = ModelverseClient(api_key)
task_input = {"prompt": prompt}
if negative_prompt:
task_input["negative_prompt"] = negative_prompt
parameters = {
"resolution": resolution,
"size": size,
"seed": seed,
"duration": 5 # Fixed as per documentation
}
# 1. Submit the task
submit_res = mv_client.submit_task("Wan-AI/Wan2.2-T2V", task_input, parameters)
task_id = submit_res.get("output", {}).get("task_id")
if not task_id:
raise Exception(f"Failed to submit task: {submit_res.get('request_id')}")
# 2. Poll for the result
video_url = ""
while True:
status_res = mv_client.get_task_status(task_id)
task_status = status_res.get("output", {}).get("task_status")
if task_status == "Success":
video_url = status_res.get("output", {}).get("urls", [None])[0]
if not video_url:
raise Exception("Task succeeded but no video URL was returned.")
break
elif task_status == "Failure":
error_message = status_res.get("output", {}).get("error_message", "Unknown error")
raise Exception(f"Task failed: {error_message}")
elif task_status in ["Pending", "Running"]:
print(f"Task {task_id} is {task_status}, waiting...")
time.sleep(5) # Wait for 5 seconds before polling again
else:
raise Exception(f"Unknown task status: {task_status}")
return (video_url, task_id)
NODE_CLASS_MAPPINGS = {
"Modelverse_WanAIT2V": Modelverse_WanAIT2V
}
NODE_DISPLAY_NAME_MAPPINGS = {
"Modelverse_WanAIT2V": "Modelverse Wan-AI T2V"
}
+3 -3
View File
@@ -1,9 +1,9 @@
[project]
name = "ucloud-comfyui"
description = "Run local and API LLMs with features including Gemini 2.0 image generation, DeepSeek R1, QwenVL 2.5, QWQ 32B, Ollama, LlamaCPP, LM Studio, KoboldCPP, TextGen, and Transformers. Connect via APIs to Anthropic, Groq, OpenAI, Google Gemini, Mistral, and xAI. Create your own character assistants with custom system prompts, presets, and much more."
version = "1.0.3"
description = "This is a custom node for ComfyUI that allows you to use the UCloud API directly in ComfyUI. UCloud is a high-performance AI computing platform providing powerful image generation models and industry-leading inference speeds. It offers a wide range of AI model options, including popular text-to-image and image editing models, ready to use without local GPU setup."
version = "1.0.9"
license = {file = "LICENSE"}
dependencies = ["requests", "pillow"]
dependencies = ["requests", "pillow", "numpy", "openai"]
[project.urls]
Repository = "https://github.com/ucloud/comfyui-plugin"
+3
View File
@@ -1,2 +1,5 @@
requests >= 2.32.4
pillow >= 11
numpy >= 2.0
openai >= 1.107.0
GitPython==3.1.45
+136
View File
@@ -0,0 +1,136 @@
.mv-secret-overlay {
position: fixed;
inset: 0;
z-index: 9999;
display: flex;
align-items: center;
justify-content: center;
background: rgba(0, 0, 0, 0.58);
}
.mv-secret-dialog {
width: min(760px, 92vw);
max-height: 82vh;
padding: 18px;
display: flex;
flex-direction: column;
gap: 12px;
color: #f0f0f0;
background: #171717;
border: 1px solid #3f3f3f;
border-radius: 8px;
box-shadow: 0 18px 60px rgba(0, 0, 0, 0.45);
font-family: sans-serif;
}
.mv-secret-header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.mv-secret-title {
margin: 0;
font-size: 16px;
font-weight: 600;
}
.mv-secret-close {
border: 0;
color: #bdbdbd;
background: transparent;
font-size: 18px;
line-height: 1;
cursor: pointer;
}
.mv-secret-rows {
display: flex;
flex-direction: column;
gap: 8px;
overflow-y: auto;
}
.mv-secret-row {
display: grid;
grid-template-columns: minmax(130px, 0.7fr) minmax(180px, 1.3fr) auto auto;
gap: 8px;
align-items: center;
}
.mv-secret-input {
min-width: 0;
height: 30px;
padding: 5px 8px;
border: 1px solid #555;
border-radius: 4px;
color: #eee;
background: #262626;
font-size: 13px;
outline: none;
}
.mv-secret-input:focus {
border-color: #7d8cff;
}
.mv-secret-button,
.mv-secret-add {
height: 30px;
padding: 5px 9px;
border-radius: 4px;
color: #eee;
font-size: 13px;
cursor: pointer;
white-space: nowrap;
}
.mv-secret-save {
background: #2f6336;
border: 1px solid #4c9b55;
}
.mv-secret-delete {
background: #653131;
border: 1px solid #a44;
}
.mv-secret-add {
width: 100%;
text-align: left;
color: #ccc;
background: #222;
border: 1px dashed #5a5a5a;
}
.mv-secret-save-notice {
margin-top: 4px;
min-height: 18px;
color: #ff4d4f;
font-size: 13px;
font-weight: 600;
text-align: center;
opacity: 0;
transition: opacity 0.4s ease;
pointer-events: none;
}
.mv-secret-save-notice-visible {
opacity: 1;
}
.mv-secret-save-notice-fade {
opacity: 0;
}
@media (max-width: 640px) {
.mv-secret-row {
grid-template-columns: 1fr;
}
.mv-secret-button,
.mv-secret-add {
width: 100%;
}
}
+196
View File
@@ -0,0 +1,196 @@
import { app } from "../../scripts/app.js";
import { api } from "../../scripts/api.js";
const cssUrl = new URL("./modelverse_secrets.css", import.meta.url);
if (!document.querySelector(`link[href="${cssUrl}"]`)) {
document.head.appendChild(Object.assign(document.createElement("link"), { rel: "stylesheet", href: cssUrl }));
}
const NODE_NAME = "UCloud ModelVerse Secret Client";
const WIDGET_NAME = "secret";
const ENDPOINT = "/modelverse-secrets";
let cachedSecretNames = [];
function el(tag, props = {}, ...children) {
const node = Object.assign(document.createElement(tag), props);
node.append(...children);
return node;
}
async function fetchSecrets() {
const response = await api.fetchApi(ENDPOINT, { method: "GET" });
return await response.json();
}
async function refreshSecretNamesCache() {
const secrets = await fetchSecrets();
cachedSecretNames = Object.keys(secrets);
return cachedSecretNames;
}
async function refreshSecretDropdowns(preferredValue) {
const names = await refreshSecretNamesCache();
for (const node of app.graph._nodes ?? []) {
if (node.comfyClass !== NODE_NAME && node.type !== NODE_NAME) {
continue;
}
const widget = node.widgets?.find((item) => item.name === WIDGET_NAME);
if (!widget) {
continue;
}
widget.options.values = names.slice();
if (preferredValue && names.includes(preferredValue)) {
widget.value = preferredValue;
} else if (names.length && !names.includes(widget.value)) {
widget.value = names[0];
} else if (!names.length) {
widget.value = "";
}
if (typeof widget.callback === "function") {
try { widget.callback(widget.value); } catch (e) { /* noop */ }
}
node.setDirtyCanvas?.(true, true);
}
app.graph?.setDirtyCanvas?.(true, true);
}
function buildSecretRow(name = "", value = "", isNew = false, onSaved) {
let originalName = name;
const nameInput = el("input", { className: "mv-secret-input", placeholder: "Secret name", value: name });
const valueInput = el("input", { className: "mv-secret-input", placeholder: "API key", value, type: "password", autocomplete: "off" });
const saveButton = el("button", { className: "mv-secret-button mv-secret-save", textContent: isNew ? "Add" : "Save" });
const deleteButton = el("button", { className: "mv-secret-button mv-secret-delete", textContent: "Delete" });
const row = el("div", { className: "mv-secret-row" }, nameInput, valueInput, saveButton, deleteButton);
saveButton.onclick = async () => {
const key = nameInput.value.trim();
if (!key) {
nameInput.focus();
return;
}
const response = await api.fetchApi(ENDPOINT, {
method: "POST",
body: JSON.stringify({ key, value: valueInput.value }),
});
if (!response.ok) {
return;
}
if (!isNew && originalName && originalName !== key) {
await api.fetchApi(`${ENDPOINT}/${encodeURIComponent(originalName)}`, { method: "DELETE" });
}
originalName = key;
await refreshSecretDropdowns(key);
if (typeof onSaved === "function") {
onSaved();
}
if (isNew) {
row.replaceWith(buildSecretRow(key, valueInput.value, false, onSaved));
return;
}
saveButton.textContent = "Saved";
setTimeout(() => {
saveButton.textContent = "Save";
}, 1200);
};
deleteButton.onclick = async () => {
if (!originalName) {
row.remove();
return;
}
const response = await api.fetchApi(`${ENDPOINT}/${encodeURIComponent(originalName)}`, { method: "DELETE" });
if (response.ok) {
row.remove();
await refreshSecretDropdowns();
}
};
return row;
}
function createSecretsModal(secrets) {
const saveNotice = el("div", { className: "mv-secret-save-notice" });
const showSaveNotice = () => {
saveNotice.textContent = "Once added, there is a 10-second delay before the item shows up in the dropdown.";
saveNotice.classList.remove("mv-secret-save-notice-fade");
void saveNotice.offsetWidth;
saveNotice.classList.add("mv-secret-save-notice-visible");
clearTimeout(saveNotice._hideTimer);
saveNotice._hideTimer = setTimeout(() => {
saveNotice.classList.add("mv-secret-save-notice-fade");
}, 10000);
};
const rows = el(
"div",
{ className: "mv-secret-rows" },
...Object.entries(secrets).map(([name, value]) => buildSecretRow(name, value, false, showSaveNotice)),
);
const addButton = el("button", {
className: "mv-secret-add",
textContent: "+ Add Secret",
onclick: () => {
const row = buildSecretRow("", "", true, showSaveNotice);
rows.appendChild(row);
row.querySelector("input")?.focus();
},
});
const closeButton = el("button", { className: "mv-secret-close", textContent: "x" });
const overlay = el("div", { className: "mv-secret-overlay" },
el("div", { className: "mv-secret-dialog" },
el("div", { className: "mv-secret-header" },
el("h3", { className: "mv-secret-title", textContent: "Modelverse Secrets" }),
closeButton,
),
rows,
addButton,
saveNotice,
),
);
closeButton.onclick = () => overlay.remove();
overlay.addEventListener("click", (event) => {
if (event.target === overlay) {
overlay.remove();
}
});
return overlay;
}
app.registerExtension({
name: "compshare.modelverse_secrets",
async nodeCreated(node) {
if (node.comfyClass !== NODE_NAME && node.type !== NODE_NAME) {
return;
}
const names = await refreshSecretNamesCache();
const index = node.widgets?.findIndex((item) => item.name === WIDGET_NAME) ?? -1;
const savedValue = index >= 0 ? node.widgets[index].value : undefined;
if (index >= 0) {
node.widgets.splice(index, 1);
}
const defaultValue = savedValue && names.includes(savedValue) ? savedValue : (names[0] ?? "");
const comboOptions = {
get values() {
return cachedSecretNames.slice();
},
set values(v) { /* ignore external writes; cache is the source of truth */ },
};
node.addWidget("combo", WIDGET_NAME, defaultValue, () => {}, comboOptions);
node.addWidget("button", "Edit Secrets", null, async () => {
await refreshSecretNamesCache();
document.body.appendChild(createSecretsModal(await fetchSecrets()));
});
},
});