Initial commit: ComfyUI BytePlus API Integration

- Video generation nodes (Seedance Lite & Pro)
- Image generation nodes (Seedream & SeedEdit)
- Video URL to frames utility node
- Comprehensive README with usage guide
- Base64 image handling (no external hosting required)
This commit is contained in:
pixelworld AI
2025-09-04 11:30:01 -05:00
commit 309ee84f56
10 changed files with 1453 additions and 0 deletions
+88
View File
@@ -0,0 +1,88 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Virtual environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~
# OS files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# ComfyUI specific
ComfyUI-fal-API/
ComfyUI-KLingAI-API/
# Temporary files
*.tmp
*.temp
# Log files
*.log
# Config files with sensitive data (optional - user can uncomment if needed)
# config.ini
+161
View File
@@ -0,0 +1,161 @@
# ComfyUI BytePlus API Integration
This ComfyUI custom node provides comprehensive integration with BytePlus (ByteDance) AI APIs, including both video generation (Seedance) and image generation (Seedream/SeedEdit) models.
## Features
### Video Generation (Seedance)
- **Text-to-Video Generation**: Create videos from text descriptions
- **Image-to-Video Generation**: Generate videos using input images as first frames
- **First+Last Frame Generation**: Create videos between two specific frames
- **Reference Images Generation**: Generate videos based on multiple reference images
- **High-Quality Pro Models**: Access to BytePlus's premium Seedance Pro model
### Image Generation
- **Text-to-Image Generation**: Create high-quality images from text prompts using Seedream 3.0
- **Image-to-Image Editing**: Edit and transform existing images with text descriptions using SeedEdit 3.0
- **Frame Extraction**: Convert video URLs to individual frames for further processing
## Installation
1. Clone this repository into your ComfyUI custom_nodes directory:
```bash
cd ComfyUI/custom_nodes/
git clone https://github.com/FloyoAI/ComfyUI-BytePlus-API.git
```
2. Install requirements:
```bash
cd ComfyUI-BytePlus-API
pip install -r requirements.txt
```
3. Configure your BytePlus API key:
- Get your API key from: https://www.byteplus.com/
- Edit `config.ini` and replace `<your_byteplus_api_key_here>` with your actual API key
- Or set the environment variable: `export BYTEPLUS_API_KEY="your_key_here"`
4. Restart ComfyUI
## Available Nodes
### Video Generation Nodes (BytePlus/VideoGeneration)
- **Seedance Lite Text-to-Video**: Generate videos from text prompts only
- Input: Text prompt, resolution, ratio, duration, FPS, seed, camera settings
- Output: Video URL (STRING)
- Use case: Create videos from scratch using descriptive text
- **Seedance Lite Image-to-Video**: Generate videos from a starting image
- Input: Text prompt, input image, resolution, ratio, duration, FPS, seed, camera settings
- Output: Video URL (STRING)
- Use case: Animate static images or create videos with specific starting frames
- **Seedance Lite First+Last Frame**: Generate videos between two keyframes
- Input: Text prompt, first frame image, last frame image, resolution, ratio, duration
- Output: Video URL (STRING)
- Use case: Create smooth transitions between two specific images
- **Seedance Lite Reference Images**: Generate videos using multiple reference images
- Input: Text prompt, 2-4 reference images, resolution, ratio, duration
- Output: Video URL (STRING)
- Use case: Generate videos that maintain style/appearance consistency with reference materials
- **Seedance Pro Text/Image-to-Video**: High-quality video generation
- Input: Text prompt, mode selection, optional input image, resolution, ratio, duration
- Output: Video URL (STRING)
- Use case: Premium quality video generation for professional use
### Image Generation Nodes (BytePlus/ImageGeneration)
- **Seedream Text-to-Image**: Generate images from text descriptions
- Input: Text prompt, size selection, optional seed
- Output: Generated image (IMAGE)
- Use case: Create high-quality images from detailed text descriptions
- **SeedEdit Image-to-Image**: Edit and transform existing images
- Input: Text prompt, input image, size selection, guidance scale, optional seed
- Output: Edited image (IMAGE)
- Use case: Modify existing images based on text instructions (change colors, add/remove objects, style transfer)
### Utility Nodes (BytePlus/Video)
- **Video URL to Frames**: Extract all frames from a video URL
- Input: Video URL (STRING)
- Output: Frame batch (IMAGE), frame count (INT), FPS (FLOAT)
- Use case: Convert generated videos into individual frames for further image processing
## Usage Guide
### Video Generation Workflow
1. **Choose the appropriate video node** based on your input type:
- Text only → Seedance Lite Text-to-Video
- Single image → Seedance Lite Image-to-Video
- Two images → Seedance Lite First+Last Frame
- Multiple references → Seedance Lite Reference Images
- High quality → Seedance Pro
2. **Configure parameters**:
- **Prompt**: Describe what you want in the video
- **Resolution**: 480p, 720p, or 1080p
- **Ratio**: Aspect ratio (16:9, 1:1, 9:16, etc.)
- **Duration**: 3-12 seconds
3. **Connect the output** to other nodes:
- Use output URL directly with video preview nodes
- Connect to "Video URL to Frames" to extract individual frames
- Save video using external video saving nodes
### Image Generation Workflow
1. **For new images**: Use Seedream Text-to-Image
- Write detailed prompts for best results
- Experiment with different sizes for various use cases
- Use seed values for reproducible results
2. **For image editing**: Use SeedEdit Image-to-Image
- Load your source image
- Describe the changes you want in the prompt
- Adjust guidance scale (higher = more prompt adherence)
- Size can be "adaptive" to maintain original dimensions
3. **Process results**:
- Images output as standard ComfyUI IMAGE tensors
- Compatible with all ComfyUI image processing nodes
- Can be saved, edited, or used as input for video generation
## Requirements
- ComfyUI
- Python 3.8+
- BytePlus API key
- Required packages (see requirements.txt)
## Key Features
- **No External Dependencies**: Base64 image conversion means no external hosting required
- **Automatic Task Polling**: Video generation waits until completion automatically
- **Comprehensive Error Handling**: Graceful degradation with detailed error messages
- **ComfyUI Integration**:
- Video outputs as URL strings compatible with video preview nodes
- Image outputs as standard IMAGE tensors compatible with all image nodes
- **Flexible Sizing**: Support for various aspect ratios and resolutions
- **Seed Control**: Reproducible results with optional seed parameters
## Tips for Best Results
### Video Generation
- **Be specific in prompts**: Include details about movement, camera angles, and style
- **Use appropriate ratios**: Match your intended use case (16:9 for landscape, 9:16 for mobile, etc.)
- **Leverage reference images**: For consistent character/style appearance across videos
### Image Generation
- **Detailed prompts work better**: Include style, lighting, composition details
- **Use SeedEdit for modifications**: Better than text-to-image for specific edits
- **Guidance scale tuning**: Start with default 5.5, increase for more prompt adherence
- **Size selection**: Use "adaptive" for SeedEdit to maintain original proportions
## Troubleshooting
- **API Key Issues**: Ensure your BytePlus API key is correctly set in config.ini or environment variables
- **Empty Results**: Check ComfyUI console for detailed error messages
- **Image Quality**: Try different prompts, seeds, or guidance scale values for better results
+23
View File
@@ -0,0 +1,23 @@
import importlib
import importlib.util
node_list = [
"byteplus_video_node",
"video_to_frames_node",
"byteplus_image_node",
]
NODE_CLASS_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS = {}
for module_name in node_list:
imported_module = importlib.import_module(f".nodes.{module_name}", __name__)
NODE_CLASS_MAPPINGS = {**NODE_CLASS_MAPPINGS, **imported_module.NODE_CLASS_MAPPINGS}
NODE_DISPLAY_NAME_MAPPINGS = {
**NODE_DISPLAY_NAME_MAPPINGS,
**imported_module.NODE_DISPLAY_NAME_MAPPINGS,
}
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+2
View File
@@ -0,0 +1,2 @@
[API]
BYTEPLUS_API_KEY = <your_byteplus_api_key_here>
+209
View File
@@ -0,0 +1,209 @@
import io
import requests
import torch
import numpy as np
from PIL import Image
from .byteplus_utils import BytePlusConfig, BytePlusImageUtils
class BytePlusResultProcessor:
"""Utility functions for processing BytePlus image generation results."""
@staticmethod
def process_image_result(result):
"""Process BytePlus image generation result and return tensor."""
try:
images = []
# BytePlus API returns data array with url field
for img_info in result["data"]:
img_url = img_info["url"]
img_response = requests.get(img_url)
img = Image.open(io.BytesIO(img_response.content))
img_array = np.array(img).astype(np.float32) / 255.0
images.append(img_array)
# Stack the images along a new first dimension
stacked_images = np.stack(images, axis=0)
# Convert to PyTorch tensor
img_tensor = torch.from_numpy(stacked_images)
return (img_tensor,)
except Exception as e:
print(f"Error processing BytePlus image result: {str(e)}")
return BytePlusResultProcessor.create_blank_image()
@staticmethod
def create_blank_image():
"""Create a blank black image tensor."""
blank_img = Image.new("RGB", (512, 512), color="black")
img_array = np.array(blank_img).astype(np.float32) / 255.0
img_tensor = torch.from_numpy(img_array)[None,]
return (img_tensor,)
class BytePlusImageApiHandler:
"""Utility functions for BytePlus image API interactions."""
@staticmethod
def generate_image(model, arguments):
"""Generate image using BytePlus API."""
config = BytePlusConfig()
headers = {
"Authorization": f"Bearer {config.get_api_key()}",
"Content-Type": "application/json"
}
payload = {
"model": model,
**arguments
}
print(f"Making BytePlus image request to: {config.get_base_url()}/images/generations")
print(f"Model: {model}")
print(f"Requested batch size (n): {payload.get('n', 1)}")
print(f"Payload: {payload}")
try:
response = requests.post(
f"{config.get_base_url()}/images/generations",
headers=headers,
json=payload,
timeout=60
)
if response.status_code == 200:
result = response.json()
print(f"BytePlus image generation successful: {result}")
return result
else:
print(f"BytePlus image generation error: {response.status_code} - {response.text}")
raise Exception(f"API request failed: {response.status_code} - {response.text}")
except Exception as e:
print(f"Error in BytePlus image generation: {str(e)}")
raise e
@staticmethod
def handle_image_generation_error(model_name, error):
"""Handle image generation errors consistently."""
print(f"Error generating image with {model_name}: {str(error)}")
return BytePlusResultProcessor.create_blank_image()
class BytePlusSeedreamTextToImageNode:
"""BytePlus Seedream Text-to-Image Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"size": (["1024x1024", "1152x896", "896x1152", "1216x832", "832x1216",
"1344x768", "768x1344", "1536x640", "640x1536"], {"default": "1024x1024"}),
},
"optional": {
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
},
}
RETURN_TYPES = ("IMAGE",)
FUNCTION = "generate_image"
CATEGORY = "BytePlus/ImageGeneration"
def generate_image(self, prompt, size, seed=-1):
try:
arguments = {
"prompt": prompt,
"n": 1,
"size": size,
"response_format": "url",
"watermark": False, # Always false, not exposed in UI
}
# Only add seed if it's not -1 (random)
if seed != -1:
arguments["seed"] = seed
result = BytePlusImageApiHandler.generate_image(
"seedream-3-0-t2i-250415",
arguments
)
return BytePlusResultProcessor.process_image_result(result)
except Exception as e:
return BytePlusImageApiHandler.handle_image_generation_error(
"BytePlus Seedream Text-to-Image", str(e)
)
class BytePlusSeedEditImageToImageNode:
"""BytePlus SeedEdit Image-to-Image Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"image": ("IMAGE",),
"size": (["adaptive", "1024x1024", "1152x896", "896x1152", "1216x832", "832x1216",
"1344x768", "768x1344", "1536x640", "640x1536"], {"default": "adaptive"}),
},
"optional": {
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
"guidance_scale": ("FLOAT", {"default": 5.5, "min": 1.0, "max": 20.0, "step": 0.1}),
},
}
RETURN_TYPES = ("IMAGE",)
FUNCTION = "generate_image"
CATEGORY = "BytePlus/ImageGeneration"
def generate_image(self, prompt, image, size, seed=-1, guidance_scale=5.5):
try:
# Convert image to base64
image_base64 = BytePlusImageUtils.image_to_base64(image)
if not image_base64:
return BytePlusImageApiHandler.handle_image_generation_error(
"BytePlus SeedEdit Image-to-Image", "Failed to convert image to base64"
)
arguments = {
"prompt": prompt,
"image": image_base64,
"n": 1,
"size": size,
"guidance_scale": guidance_scale,
"response_format": "url",
"watermark": False, # Always false, not exposed in UI
}
# Only add seed if it's not -1 (random)
if seed != -1:
arguments["seed"] = seed
result = BytePlusImageApiHandler.generate_image(
"seededit-3-0-i2i-250628",
arguments
)
return BytePlusResultProcessor.process_image_result(result)
except Exception as e:
return BytePlusImageApiHandler.handle_image_generation_error(
"BytePlus SeedEdit Image-to-Image", str(e)
)
# Node class mappings
NODE_CLASS_MAPPINGS = {
"BytePlusSeedreamTextToImage": BytePlusSeedreamTextToImageNode,
"BytePlusSeedEditImageToImage": BytePlusSeedEditImageToImageNode,
}
# Node display name mappings
NODE_DISPLAY_NAME_MAPPINGS = {
"BytePlusSeedreamTextToImage": "Seedream Text-to-Image",
"BytePlusSeedEditImageToImage": "SeedEdit Image-to-Image",
}
+304
View File
@@ -0,0 +1,304 @@
import base64
import configparser
import io
import json
import os
import time
import tempfile
from typing import Optional, Dict, Any
import numpy as np
import requests
import torch
from PIL import Image
class BytePlusConfig:
"""Singleton class to handle BytePlus configuration and client setup."""
_instance = None
_api_key = None
_base_url = "https://ark.ap-southeast.bytepluses.com/api/v3"
def __new__(cls):
if cls._instance is None:
cls._instance = super(BytePlusConfig, cls).__new__(cls)
cls._instance._initialize()
return cls._instance
def _initialize(self):
"""Initialize configuration and API key."""
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
config_path = os.path.join(parent_dir, "config.ini")
config = configparser.ConfigParser()
config.read(config_path)
try:
if os.environ.get("BYTEPLUS_API_KEY") is not None:
print("BYTEPLUS_API_KEY found in environment variables")
self._api_key = os.environ["BYTEPLUS_API_KEY"]
else:
print("BYTEPLUS_API_KEY not found in environment variables")
self._api_key = config["API"]["BYTEPLUS_API_KEY"]
print("BYTEPLUS_API_KEY found in config.ini")
os.environ["BYTEPLUS_API_KEY"] = self._api_key
print("BYTEPLUS_API_KEY set in environment variables")
# Check if BytePlus key is the default placeholder
if self._api_key == "<your_byteplus_api_key_here>":
print("WARNING: You are using the default BytePlus API key placeholder!")
print("Please set your actual BytePlus API key in either:")
print("1. The config.ini file under [API] section")
print("2. Or as an environment variable named BYTEPLUS_API_KEY")
print("Get your API key from: https://console.volcengine.com/ark/region:ap-southeast-1/apiKey")
except KeyError:
print("Error: BYTEPLUS_API_KEY not found in config.ini or environment variables")
def get_api_key(self):
"""Get the BytePlus API key."""
return self._api_key
def get_base_url(self):
"""Get the BytePlus API base URL."""
return self._base_url
class BytePlusImageUtils:
"""Utility functions for image processing specific to BytePlus API."""
@staticmethod
def tensor_to_pil(image):
"""Convert image tensor to PIL Image."""
try:
# Convert the image tensor to a numpy array
if isinstance(image, torch.Tensor):
image_np = image.cpu().numpy()
else:
image_np = np.array(image)
# Ensure the image is in the correct format (H, W, C)
if image_np.ndim == 4:
image_np = image_np.squeeze(0) # Remove batch dimension if present
if image_np.ndim == 2:
image_np = np.stack([image_np] * 3, axis=-1) # Convert grayscale to RGB
elif image_np.shape[0] == 3:
image_np = np.transpose(
image_np, (1, 2, 0)
) # Change from (C, H, W) to (H, W, C)
# Normalize the image data to 0-255 range
if image_np.dtype == np.float32 or image_np.dtype == np.float64:
image_np = (image_np * 255).astype(np.uint8)
# Convert to PIL Image
return Image.fromarray(image_np)
except Exception as e:
print(f"Error converting tensor to PIL: {str(e)}")
return None
@staticmethod
def image_to_base64(image):
"""Convert image tensor to base64 string format for BytePlus API."""
try:
pil_image = BytePlusImageUtils.tensor_to_pil(image)
if not pil_image:
return None
# Convert to base64
buffered = io.BytesIO()
pil_image.save(buffered, format="PNG")
img_str = base64.b64encode(buffered.getvalue()).decode()
# Format for BytePlus API
return f"data:image/png;base64,{img_str}"
except Exception as e:
print(f"Error converting image to base64: {str(e)}")
return None
class BytePlusApiHandler:
"""Utility functions for BytePlus API interactions."""
@staticmethod
def create_video_generation_task(model: str, content: list, callback_url: Optional[str] = None) -> Optional[str]:
"""Create a video generation task and return task ID."""
config = BytePlusConfig()
headers = {
"Authorization": f"Bearer {config.get_api_key()}",
"Content-Type": "application/json"
}
payload = {
"model": model,
"content": content
}
print(f"Making request to: {config.get_base_url()}/contents/generations/tasks")
print(f"Model: {model}")
print(f"API Key length: {len(config.get_api_key()) if config.get_api_key() else 0}")
print(f"API Key starts with: {config.get_api_key()[:10] if config.get_api_key() else 'None'}...")
print(f"Headers: {headers}")
print(f"Payload: {payload}")
if callback_url:
payload["callback_url"] = callback_url
try:
response = requests.post(
f"{config.get_base_url()}/contents/generations/tasks",
headers=headers,
json=payload,
timeout=30
)
if response.status_code == 200:
result = response.json()
return result.get("id")
else:
print(f"Error creating task: {response.status_code} - {response.text}")
return None
except Exception as e:
print(f"Error creating video generation task: {str(e)}")
return None
@staticmethod
def query_task_status(task_id: str) -> Optional[Dict[str, Any]]:
"""Query the status of a video generation task."""
config = BytePlusConfig()
headers = {
"Authorization": f"Bearer {config.get_api_key()}",
"Content-Type": "application/json"
}
try:
response = requests.get(
f"{config.get_base_url()}/contents/generations/tasks/{task_id}",
headers=headers,
timeout=30
)
if response.status_code == 200:
return response.json()
else:
print(f"Error querying task: {response.status_code} - {response.text}")
return None
except Exception as e:
print(f"Error querying task status: {str(e)}")
return None
@staticmethod
def wait_for_completion(task_id: str, max_wait_time: int = 300, poll_interval: int = 5) -> Optional[str]:
"""Wait for task completion and return video URL."""
start_time = time.time()
while time.time() - start_time < max_wait_time:
result = BytePlusApiHandler.query_task_status(task_id)
if not result:
print(f"Failed to query task status for {task_id}")
return None
status = result.get("status")
print(f"Task {task_id} status: {status}")
if status == "succeeded":
# Debug: Print the full response to understand the structure
print(f"Full successful response: {result}")
# Look for video URL in the result - try multiple possible paths
if "content" in result and "video_url" in result["content"]:
print(f"Found video_url in content: {result['content']['video_url']}")
return result["content"]["video_url"]
elif "video_url" in result:
print(f"Found video_url at root level: {result['video_url']}")
return result["video_url"]
elif "result" in result and "video_url" in result["result"]:
print(f"Found video_url in result: {result['result']['video_url']}")
return result["result"]["video_url"]
elif "data" in result and "video_url" in result["data"]:
print(f"Found video_url in data: {result['data']['video_url']}")
return result["data"]["video_url"]
elif "output" in result and "video_url" in result["output"]:
print(f"Found video_url in output: {result['output']['video_url']}")
return result["output"]["video_url"]
elif "video" in result and "url" in result["video"]:
print(f"Found url in video: {result['video']['url']}")
return result["video"]["url"]
else:
print("Task succeeded but no video URL found")
print(f"Available keys: {list(result.keys())}")
return None
elif status == "failed":
print(f"Task {task_id} failed: {result.get('error', 'Unknown error')}")
return None
elif status in ["queued", "running"]:
time.sleep(poll_interval)
else:
print(f"Unknown task status: {status}")
return None
print(f"Task {task_id} timed out after {max_wait_time} seconds")
return None
@staticmethod
def submit_and_get_result(model: str, content: list, callback_url: Optional[str] = None) -> Optional[str]:
"""Submit task and wait for completion, return video URL."""
task_id = BytePlusApiHandler.create_video_generation_task(model, content, callback_url)
if not task_id:
return None
print(f"Created BytePlus task: {task_id}")
return BytePlusApiHandler.wait_for_completion(task_id)
@staticmethod
def handle_video_generation_error(model_name: str, error: str) -> tuple:
"""Handle video generation errors consistently."""
print(f"Error generating video with {model_name}: {str(error)}")
# Return empty string instead of None to avoid URL errors
return ("",)
class BytePlusPromptBuilder:
"""Utility class for building BytePlus prompts with text commands."""
@staticmethod
def build_prompt_with_commands(
prompt: str,
resolution: str = None,
ratio: str = None,
duration: int = None,
framepersecond: int = None,
watermark: bool = None,
seed: int = None,
camerafixed: bool = None
) -> str:
"""Build prompt with text commands appended."""
commands = []
if resolution:
commands.append(f"--rs {resolution}")
if ratio:
commands.append(f"--rt {ratio}")
if duration is not None:
commands.append(f"--dur {duration}")
if framepersecond is not None:
commands.append(f"--fps {framepersecond}")
if watermark is not None:
commands.append(f"--wm {str(watermark).lower()}")
if seed is not None and seed != -1:
commands.append(f"--seed {seed}")
if camerafixed is not None:
commands.append(f"--cf {str(camerafixed).lower()}")
if commands:
return f"{prompt} {' '.join(commands)}"
return prompt
+478
View File
@@ -0,0 +1,478 @@
from .byteplus_utils import BytePlusApiHandler, BytePlusImageUtils, BytePlusPromptBuilder
class BytePlusSeedanceTextToVideoNode:
"""BytePlus Seedance Text-to-Video Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"resolution": (["480p", "720p", "1080p"], {"default": "720p"}),
"ratio": (["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"], {"default": "16:9"}),
"duration": ("INT", {"default": 5, "min": 3, "max": 12, "step": 1}),
},
"optional": {
"framepersecond": ("INT", {"default": 24, "min": 24, "max": 24, "step": 1}),
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
"camerafixed": ("BOOLEAN", {"default": False}),
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_video"
CATEGORY = "BytePlus/VideoGeneration"
def generate_video(
self,
prompt,
resolution,
ratio,
duration,
framepersecond=24,
seed=-1,
camerafixed=False
):
try:
# Build prompt with text commands
full_prompt = BytePlusPromptBuilder.build_prompt_with_commands(
prompt=prompt,
resolution=resolution,
ratio=ratio,
duration=duration,
framepersecond=framepersecond,
watermark=False,
seed=seed,
camerafixed=camerafixed
)
# Prepare content for API
content = [
{
"type": "text",
"text": full_prompt
}
]
# Submit to BytePlus API
video_url = BytePlusApiHandler.submit_and_get_result(
model="seedance-1-0-lite-t2v-250428",
content=content
)
if video_url and video_url.strip():
return (video_url,)
else:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Text-to-Video", "Failed to generate video"
)
except Exception as e:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Text-to-Video", str(e)
)
class BytePlusSeedanceImageToVideoNode:
"""BytePlus Seedance Image-to-Video Generation Node (First Frame)"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"image": ("IMAGE",),
"resolution": (["480p", "720p", "1080p"], {"default": "720p"}),
"ratio": (["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"], {"default": "adaptive"}),
"duration": ("INT", {"default": 5, "min": 3, "max": 12, "step": 1}),
},
"optional": {
"framepersecond": ("INT", {"default": 24, "min": 24, "max": 24, "step": 1}),
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
"camerafixed": ("BOOLEAN", {"default": False}),
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_video"
CATEGORY = "BytePlus/VideoGeneration"
def generate_video(
self,
prompt,
image,
resolution,
ratio,
duration,
framepersecond=24,
seed=-1,
camerafixed=False
):
try:
# Convert image to base64
image_base64 = BytePlusImageUtils.image_to_base64(image)
if not image_base64:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Image-to-Video", "Failed to convert image to base64"
)
# Build prompt with text commands
full_prompt = BytePlusPromptBuilder.build_prompt_with_commands(
prompt=prompt,
resolution=resolution,
ratio=ratio,
duration=duration,
framepersecond=framepersecond,
watermark=False,
seed=seed,
camerafixed=camerafixed
)
# Prepare content for API
content = [
{
"type": "text",
"text": full_prompt
},
{
"type": "image_url",
"image_url": {
"url": image_base64
},
"role": "first_frame"
}
]
# Submit to BytePlus API
video_url = BytePlusApiHandler.submit_and_get_result(
model="seedance-1-0-lite-i2v-250428",
content=content
)
if video_url and video_url.strip():
return (video_url,)
else:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Image-to-Video", "Failed to generate video"
)
except Exception as e:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Image-to-Video", str(e)
)
class BytePlusSeedanceFirstLastFrameNode:
"""BytePlus Seedance First+Last Frame Video Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"first_frame": ("IMAGE",),
"last_frame": ("IMAGE",),
"resolution": (["480p", "720p"], {"default": "720p"}),
"ratio": (["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"], {"default": "adaptive"}),
"duration": ("INT", {"default": 5, "min": 3, "max": 12, "step": 1}),
},
"optional": {
"framepersecond": ("INT", {"default": 24, "min": 24, "max": 24, "step": 1}),
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_video"
CATEGORY = "BytePlus/VideoGeneration"
def generate_video(
self,
prompt,
first_frame,
last_frame,
resolution,
ratio,
duration,
framepersecond=24,
seed=-1
):
try:
# Convert images to base64
first_frame_base64 = BytePlusImageUtils.image_to_base64(first_frame)
last_frame_base64 = BytePlusImageUtils.image_to_base64(last_frame)
if not first_frame_base64 or not last_frame_base64:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance First+Last Frame", "Failed to convert images to base64"
)
# Build prompt with text commands (no camerafixed for this mode)
full_prompt = BytePlusPromptBuilder.build_prompt_with_commands(
prompt=prompt,
resolution=resolution,
ratio=ratio,
duration=duration,
framepersecond=framepersecond,
watermark=False,
seed=seed
)
# Prepare content for API
content = [
{
"type": "text",
"text": full_prompt
},
{
"type": "image_url",
"image_url": {
"url": first_frame_base64
},
"role": "first_frame"
},
{
"type": "image_url",
"image_url": {
"url": last_frame_base64
},
"role": "last_frame"
}
]
# Submit to BytePlus API (only supported by seedance-1-0-lite-i2v)
video_url = BytePlusApiHandler.submit_and_get_result(
model="seedance-1-0-lite-i2v-250428",
content=content
)
if video_url and video_url.strip():
return (video_url,)
else:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance First+Last Frame", "Failed to generate video"
)
except Exception as e:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance First+Last Frame", str(e)
)
class BytePlusSeedanceReferenceImagesNode:
"""BytePlus Seedance Reference Images Video Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"reference_image_1": ("IMAGE",),
"reference_image_2": ("IMAGE",),
"resolution": (["480p", "720p"], {"default": "720p"}),
"ratio": (["16:9", "4:3", "1:1", "3:4", "9:16", "21:9"], {"default": "16:9"}),
"duration": ("INT", {"default": 5, "min": 3, "max": 12, "step": 1}),
},
"optional": {
"reference_image_3": ("IMAGE",),
"reference_image_4": ("IMAGE",),
"framepersecond": ("INT", {"default": 24, "min": 24, "max": 24, "step": 1}),
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_video"
CATEGORY = "BytePlus/VideoGeneration"
def generate_video(
self,
prompt,
reference_image_1,
reference_image_2,
resolution,
ratio,
duration,
reference_image_3=None,
reference_image_4=None,
framepersecond=24,
seed=-1
):
try:
# Convert required images to base64
ref_images = [reference_image_1, reference_image_2]
if reference_image_3 is not None:
ref_images.append(reference_image_3)
if reference_image_4 is not None:
ref_images.append(reference_image_4)
# Build prompt with text commands (no camerafixed for reference images)
full_prompt = BytePlusPromptBuilder.build_prompt_with_commands(
prompt=prompt,
resolution=resolution,
ratio=ratio,
duration=duration,
framepersecond=framepersecond,
watermark=False,
seed=seed
)
# Prepare content for API
content = [
{
"type": "text",
"text": full_prompt
}
]
# Add reference images
for i, ref_image in enumerate(ref_images):
image_base64 = BytePlusImageUtils.image_to_base64(ref_image)
if not image_base64:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Reference Images", f"Failed to convert reference image {i+1} to base64"
)
content.append({
"type": "image_url",
"image_url": {
"url": image_base64
},
"role": "reference_image"
})
# Submit to BytePlus API (only supported by seedance-1-0-lite-i2v)
video_url = BytePlusApiHandler.submit_and_get_result(
model="seedance-1-0-lite-i2v-250428",
content=content
)
if video_url and video_url.strip():
return (video_url,)
else:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Reference Images", "Failed to generate video"
)
except Exception as e:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Reference Images", str(e)
)
class BytePlusSeedanceProNode:
"""BytePlus Seedance Pro Video Generation Node"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"default": "", "multiline": True}),
"mode": (["text-to-video", "image-to-video"], {"default": "text-to-video"}),
"resolution": (["480p", "720p", "1080p"], {"default": "1080p"}),
"ratio": (["16:9", "4:3", "1:1", "3:4", "9:16", "21:9", "adaptive"], {"default": "adaptive"}),
"duration": ("INT", {"default": 5, "min": 3, "max": 12, "step": 1}),
},
"optional": {
"image": ("IMAGE",),
"framepersecond": ("INT", {"default": 24, "min": 24, "max": 24, "step": 1}),
"seed": ("INT", {"default": -1, "min": -1, "max": 2147483647}),
"camerafixed": ("BOOLEAN", {"default": False}),
},
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_video"
CATEGORY = "BytePlus/VideoGeneration"
def generate_video(
self,
prompt,
mode,
resolution,
ratio,
duration,
image=None,
framepersecond=24,
seed=-1,
camerafixed=False
):
try:
# Build prompt with text commands
full_prompt = BytePlusPromptBuilder.build_prompt_with_commands(
prompt=prompt,
resolution=resolution,
ratio=ratio,
duration=duration,
framepersecond=framepersecond,
watermark=False,
seed=seed,
camerafixed=camerafixed
)
# Prepare content for API
content = [
{
"type": "text",
"text": full_prompt
}
]
# Add image if in image-to-video mode
if mode == "image-to-video":
if image is None:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Pro", "Image is required for image-to-video mode"
)
image_base64 = BytePlusImageUtils.image_to_base64(image)
if not image_base64:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Pro", "Failed to convert image to base64"
)
content.append({
"type": "image_url",
"image_url": {
"url": image_base64
},
"role": "first_frame"
})
# Submit to BytePlus API
video_url = BytePlusApiHandler.submit_and_get_result(
model="seedance-1-0-pro-250528",
content=content
)
if video_url and video_url.strip():
return (video_url,)
else:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Pro", "Failed to generate video"
)
except Exception as e:
return BytePlusApiHandler.handle_video_generation_error(
"BytePlus Seedance Pro", str(e)
)
# Node class mappings
NODE_CLASS_MAPPINGS = {
"SeedanceLiteTextToVideo": BytePlusSeedanceTextToVideoNode,
"SeedanceLiteImageToVideo": BytePlusSeedanceImageToVideoNode,
"SeedanceLiteFirstLastFrame": BytePlusSeedanceFirstLastFrameNode,
"SeedanceLiteReferenceImages": BytePlusSeedanceReferenceImagesNode,
"SeedanceProTextImageToVideo": BytePlusSeedanceProNode,
}
# Node display name mappings
NODE_DISPLAY_NAME_MAPPINGS = {
"SeedanceLiteTextToVideo": "Seedance Lite Text-to-Video",
"SeedanceLiteImageToVideo": "Seedance Lite Image-to-Video",
"SeedanceLiteFirstLastFrame": "Seedance Lite First+Last Frame",
"SeedanceLiteReferenceImages": "Seedance Lite Reference Images",
"SeedanceProTextImageToVideo": "Seedance Pro Text/Image-to-Video",
}
+158
View File
@@ -0,0 +1,158 @@
import os
import torch
import requests
import tempfile
import numpy as np
from PIL import Image
# Try to import opencv - if not available, provide error message
try:
import cv2
OPENCV_AVAILABLE = True
except ImportError:
OPENCV_AVAILABLE = False
print("WARNING: opencv-python not installed. Video frame extraction will not work.")
print("To install: pip install opencv-python>=4.5.0")
def _fetch_video(url, stream=True):
"""Download video content from URL."""
return requests.get(url, stream=stream).content
class VideoToFrames:
"""
Simple node that extracts all frames from a video URL as a tensor batch.
Clean and focused approach without file saving complexity.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"video_url": ("STRING", {"forceInput": True}),
}
}
FUNCTION = "extract_frames"
CATEGORY = "BytePlus/Video"
RETURN_TYPES = ("IMAGE", "INT", "FLOAT")
RETURN_NAMES = ("frames", "frame_count", "fps")
def extract_frames(self, video_url):
"""
Simple function that downloads video and extracts all frames.
Args:
video_url: URL of the video to download
Returns:
frames: Extracted frames as IMAGE tensor batch
frame_count: Number of extracted frames
fps: FPS of the video
"""
# Handle case where video_url might be a list
if isinstance(video_url, list):
video_url = video_url[0]
# Initialize return values
frames_tensor = torch.zeros((1, 64, 64, 3), dtype=torch.float32) # Default empty frames
frame_count = 0
video_fps = 0.0
# Download video content
try:
print(f"Downloading video from: {video_url}")
video_content = _fetch_video(video_url)
print(f"Downloaded {len(video_content)} bytes")
except Exception as e:
print(f"Error downloading video: {str(e)}")
return (frames_tensor, frame_count, video_fps)
# Extract frames
if video_content:
try:
frames_tensor, frame_count, video_fps = self._extract_frames(video_content)
print(f"Extracted {frame_count} frames at {video_fps:.2f} fps")
except Exception as e:
print(f"Error extracting frames: {str(e)}")
frames_tensor = torch.zeros((1, 64, 64, 3), dtype=torch.float32)
frame_count = 0
video_fps = 0.0
return (frames_tensor, frame_count, video_fps)
def _extract_frames(self, video_content):
"""Extract all frames from video content as tensor batch."""
if not OPENCV_AVAILABLE:
print("Cannot extract frames: opencv-python not installed")
return torch.zeros((1, 64, 64, 3), dtype=torch.float32), 0, 0.0
# Create temporary file for OpenCV
with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as temp_file:
temp_file.write(video_content)
temp_path = temp_file.name
try:
# Open video with OpenCV
cap = cv2.VideoCapture(temp_path)
if not cap.isOpened():
raise Exception("Could not open video file")
# Get video properties
fps = cap.get(cv2.CAP_PROP_FPS)
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
frames = []
frame_count = 0
print(f"Video info: {total_frames} total frames at {fps:.2f} fps")
print(f"Extracting all frames...")
while True:
ret, frame = cap.read()
if not ret:
break
# Convert BGR to RGB
frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
# Convert to PIL Image then to tensor
pil_image = Image.fromarray(frame_rgb)
frame_array = np.array(pil_image).astype(np.float32) / 255.0
frames.append(frame_array)
frame_count += 1
cap.release()
if frames:
# Stack frames into tensor batch
frames_tensor = torch.from_numpy(np.stack(frames, axis=0))
print(f"Created tensor with shape: {frames_tensor.shape}")
else:
# Return empty tensor if no frames extracted
frames_tensor = torch.zeros((1, 64, 64, 3), dtype=torch.float32)
frame_count = 0
fps = 0.0
return frames_tensor, frame_count, fps
finally:
# Clean up temporary file
if os.path.exists(temp_path):
os.unlink(temp_path)
# Node class mappings
NODE_CLASS_MAPPINGS = {
"VideoToFrames": VideoToFrames,
}
# Node display name mappings
NODE_DISPLAY_NAME_MAPPINGS = {
"VideoToFrames": "Video URL to Frames",
}
+25
View File
@@ -0,0 +1,25 @@
[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "comfyui-byteplus-api"
version = "1.0.0"
description = "ComfyUI custom nodes for BytePlus (ByteDance) AI video generation API"
authors = [
{name = "ComfyUI BytePlus Integration"}
]
license = {text = "MIT"}
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
"requests>=2.25.0",
"torch>=1.9.0",
"pillow>=8.0.0",
"numpy>=1.20.0",
]
[project.urls]
Homepage = "https://github.com/your-repo/ComfyUI-BytePlus-API"
Repository = "https://github.com/your-repo/ComfyUI-BytePlus-API"
Issues = "https://github.com/your-repo/ComfyUI-BytePlus-API/issues"
+5
View File
@@ -0,0 +1,5 @@
requests>=2.25.0
torch>=1.9.0
pillow>=8.0.0
numpy>=1.20.0
opencv-python>=4.5.0