diff --git a/README.md b/README.md index 6ea2976..7500ede 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,25 @@ # ComfyUI-ExternalAPI-Helpers -A collection of powerful custom nodes for ComfyUI that connect your local workflows to closed-source AI models via their APIs. Use Google's Gemini, Imagen, Veo, OpenAI's GPT-Image-1, and Black Forest Labs' FLUX models directly within ComfyUI. +A collection of powerful custom nodes for ComfyUI that connect your local workflows to closed-source AI models via their APIs. Use Google's Gemini, Imagen, Veo 3.1, OpenAI's GPT-6 and GPT Image 2.5, ElevenLabs v4, and Black Forest Labs' FLUX models directly within ComfyUI. ## Key Features * **FLUX Kontext Pro & Max:** Image-to-image transformations using the FLUX models via the Replicate API. * **Flux.2 (Replicate):** Generate images using the latest FLUX.2 models (Pro, Max, Dev) via Replicate. -* **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models. Now supports audio input. -* **Gemini Speaker Diarization:** Separate audio into different speaker tracks using Gemini. -* **GPT Image Edit:** OpenAI's `gpt-image-1` for prompt-based image editing and inpainting. Simply mask an area and describe the change you want to see. -* **OpenAI LLM:** Access OpenAI's powerful language models (GPT-4, GPT-5, o1, etc.) for text generation and reasoning. +* **Gemini Chat:** Google's frontier multimodal models (Gemini 3.8 Flash, 3.6 Flash, 3.5 Flash-Lite, 3.1 Pro). Analyze images, process audio waveforms, and generate responses with dynamic thinking controls. +* **Gemini Speaker Diarization:** Separate audio into individual speaker tracks with high-precision timestamping using Gemini 3.8. +* **GPT Image Edit:** Prompt-based image editing and inpainting with OpenAI's latest `gpt-image-2.5-sunburst` and `gpt-image-2.5-flare` alongside GPT Image 2. +* **OpenAI LLM:** Access OpenAI's frontier reasoning and generation models (GPT-6, GPT-5.6 family, GPT-5.5, GPT-5.4) using the native Responses API with selectable reasoning effort (`none`, `low`, `medium`, `high`). +* **OpenAI Text-to-Speech:** Generate speech with OpenAI's TTS models (`gpt-4o-mini-tts`, `tts-1`, `tts-1-hd`) featuring all 13 built-in voices and natural language tone/style instructions. +* **ElevenLabs TTS:** Generate speech using ElevenLabs' newest **Eleven v4** and **Eleven v4 Turbo** models with automatic parameter branching, alongside legacy v3 and multilingual engines. +* **Gemini 3.8 TTS:** Next-gen speech synthesis powered by Gemini 3.8 with access to Google's Extended Voice Library (2,000+ voices like *Bodi*, *Lumi*, *Koda*), native `speech_metadata` style direction, and inline vocal tags (``, ``, ``, ``, ``). +* **Gemini TTS (Legacy):** Dedicated backwards-compatible node for earlier Gemini 2.5 and 3.1 celestial voice endpoints. +* **Nano Banana:** Google's native image generation and editing using the latest **`gemini-nano-banana-2.1`**, featuring 1K/2K/4K resolution control and multi-image consistency. +* **Veo Video Generator (Gemini API & Vertex AI):** Generate video clips using Google's **Veo 3.1** family (`veo-3.1-generate-preview`, `veo-3.1-fast`, `veo-3.1-lite` / Vertex `001` releases) with native synchronized audio, 1080p HD support, and flexible durations. +* **Google Imagen Generator & Edit:** Create and edit images with Google's Imagen models, with support for Vertex AI. * **Groq LLM:** Fast text generation using Groq-hosted models, with optional image input for supported vision models. * **Groq Orpheus TTS:** Generate speech using Groq-hosted Orpheus text-to-speech models. -* **OpenAI Text-to-Speech:** Generate high-quality speech using OpenAI's TTS models. -* **Google Imagen Generator & Edit:** Create and edit images with Google's Imagen models, with support for Vertex AI. -* **Nano Banana:** A creative image generation node using a specialized Gemini model. -* **Veo Video Generator:** Generate high-quality video clips from text prompts using Google's Veo model via Vertex AI or the Gemini API. -* **ElevenLabs TTS:** Generate high-quality speech from text using ElevenLabs' diverse range of voices and models. -* **Gemini TTS:** Create speech from text using Google's Gemini models. -* **Tripo Text-to-3D:** Generate 3D models from text prompts using Tripo AI's advanced 3D generation API. -* **Tripo Image-to-3D:** Convert images into detailed 3D models using Tripo AI's image-to-model capability. +* **Tripo Text-to-3D & Image-to-3D:** Generate and convert images into production 3D models using Tripo AI. --- @@ -34,208 +34,146 @@ A collection of powerful custom nodes for ComfyUI that connect your local workfl ```bash git clone https://github.com/Aryan185/ComfyUI-ExternalAPI-Helpers.git ``` - -4. Install the required Python packages. Navigate into the newly cloned directory and use pip to install the dependencies: +4. Install the required Python packages: ```bash cd ComfyUI-ExternalAPI-Helpers pip install -r requirements.txt ``` -5. **Restart ComfyUI.** After restarting, you should find the new nodes in the "Add Node" menu. +5. **Restart ComfyUI.** The nodes will be available in the "Add Node" menu. --- ## 🔑 Prerequisites: API Keys -All nodes in this collection require API keys to function. - -* **FLUX Nodes (Replicate):** You will need a [Replicate API Token](https://replicate.com/account/api-tokens). -* **Gemini, Imagen, Nano Banana, Gemini TTS, Gemini Diarization, and Veo (Gemini API) Nodes:** You will need a [Google AI Studio API Key](https://aistudio.google.com/app/api-keys). -* **OpenAI Nodes (GPT Image Edit, OpenAI LLM, OpenAI TTS):** You will need an [OpenAI API Key](https://platform.openai.com/api-keys). -* **Groq Nodes (LLM, Orpheus TTS):** You will need a [Groq API Key](https://console.groq.com/keys). -* **ElevenLabs TTS Node:** You will need an [ElevenLabs API Key](https://elevenlabs.io/). -* **Tripo Nodes (Text-to-3D, Image-to-3D):** You will need a [Tripo API Key](https://tripo3d.ai/). -* **Vertex AI Nodes (Imagen Edit, Veo Vertex AI):** You will need a Google Cloud Project ID, a service account with appropriate permissions, and the location for the resources. - -You can paste your key directly into the `api_key` field on the corresponding node. For Vertex AI nodes, you will need to provide the project ID, location, and paste your service account JSON content. +* **FLUX Nodes (Replicate):** [Replicate API Token](https://replicate.com/account/api-tokens). +* **Google AI Studio Nodes (Gemini Chat, Diarization, Nano Banana, Gemini 3.8 TTS, Veo Gemini API):** [Google AI Studio API Key](https://aistudio.google.com/app/api-keys). +* **OpenAI Nodes (GPT Image Edit, OpenAI LLM, OpenAI TTS):** [OpenAI API Key](https://platform.openai.com/api-keys). +* **ElevenLabs TTS Node:** [ElevenLabs API Key](https://elevenlabs.io/). +* **Groq Nodes (LLM, Orpheus TTS):** [Groq API Key](https://console.groq.com/keys). +* **Tripo Nodes (Text-to-3D, Image-to-3D):** [Tripo API Key](https://tripo3d.ai/). +* **Vertex AI Nodes (Imagen Edit, Veo Vertex AI):** Google Cloud Project ID, location, and service account JSON key. ### Using Environment Variables (.env file) -Instead of pasting API keys directly into nodes, you can store them in a `.env` file: +Instead of pasting raw API keys into every node, store them in a `.env` file in the node directory: -1. Copy `.env.example` to `.env` in the node directory -2. Fill in your API keys: - ``` - GEMINI_API_KEY=your_gemini_key_here - OPENAI_API_KEY=your_openai_key_here - GROQ_API_KEY=your_groq_key_here - XI_API_KEY=your_elevenlabs_key_here - REPLICATE_API_TOKEN=your_replicate_token_here - TRIPO_API_KEY=your_tripo_api_key_here - ``` -3. In the node's `api_key` field, enter the variable name (e.g., `GEMINI_API_KEY`) instead of the actual key +```env +GEMINI_API_KEY=your_gemini_key_here +OPENAI_API_KEY=your_openai_key_here +XI_API_KEY=your_elevenlabs_key_here +GROQ_API_KEY=your_groq_key_here +REPLICATE_API_TOKEN=your_replicate_token_here +TRIPO_API_KEY=your_tripo_api_key_here +``` -This keeps your API keys secure and makes it easy to switch between different keys or projects. +In any node's `api_key` field, you can simply type the variable name (e.g. `GEMINI_API_KEY`) instead of exposing your actual secret. --- ## 📚 Node Guide -> **Note:** All nodes include standard inputs like `api_key`, `prompt`, `model`, `temperature`, and `seed` where applicable. Only unique or notable inputs are listed below. - ### Flux Kontext Pro / Max - -Transform images based on text prompts. Ideal for applying artistic styles or making conceptual changes. - * **Category:** `image/edit` * **Key Inputs:** `image`, `aspect_ratio` (use `match_input_image` to preserve composition) * **Output:** `image` ### Flux.2 (Replicate) - -Generate images using FLUX.2 models (Pro, Max, Dev) via Replicate. - * **Category:** `image/generation` * **Key Inputs:** `image_1` to `image_5` (optional, for image-to-image tasks) * **Output:** `image` ### Gemini Chat - -Multimodal text generation with image and audio analysis capabilities. - * **Category:** `text/generation` -* **Key Inputs:** `thinking`, `thinking_budget`, `system_instruction`, `image` (optional), `audio` (optional) +* **Models:** `gemini-3.8-flash` (default), `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.1-pro-preview`, dynamic aliases. +* **Key Inputs:** `prompt`, `thinking`, `thinking_budget`, `google_search`, `system_instruction`, `image` (optional), `audio` (optional) * **Output:** `response` (text) ### Gemini Speaker Diarization - -Separate audio into different speaker tracks. - * **Category:** `audio/diarise` -* **Key Inputs:** `audio`, `num_speakers`, `thinking`, `thinking_budget` -* **Output:** `speaker_1` to `speaker_4` (audio tracks) +* **Key Inputs:** `audio`, `num_speakers`, `model`, `thinking`, `thinking_budget` +* **Output:** `speaker_1` to `speaker_4` (isolated audio tracks) ### GPT Image Edit - -Prompt-based inpainting and image editing using OpenAI's API. - -* **Category:** `image/edit` -* **Key Inputs:** `image_1` to `image_5`, `mask` (optional, white area = edit region), `background`, `quality`, `size` +* **Category:** `image/generation` +* **Models:** `gpt-image-2.5-sunburst` (default), `gpt-image-2.5-flare`, `gpt-image-2`, `gpt-image-1.5`, `gpt-image-1` +* **Key Inputs:** `prompt`, `image_1` to `image_5` (optional reference/edit inputs), `mask` (optional, white area = inpainting region), `background`, `quality`, `size` * **Output:** `image` ### OpenAI LLM - -Access OpenAI language models (GPT-4, GPT-5, o1, etc.) for text generation. - * **Category:** `text/generation` -* **Key Inputs:** `reasoning_effort` (low/medium/high), `max_output_tokens`, `system_instruction`, `image` (optional) +* **Models:** `gpt-6` (default), `gpt-6.1-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.4`, `gpt-4.1` +* **Key Inputs:** `prompt`, `reasoning_effort` (`none`, `low`, `medium`, `high`), `max_output_tokens`, `system_instruction`, `image` (optional) * **Output:** `response` (text) -### Groq LLM - -Access Groq-hosted language models for fast text generation, with optional image understanding on supported vision models. - -* **Category:** `text/generation` -* **Key Inputs:** `max_completion_tokens`, `reasoning_effort`, `system_instruction`, `image` (optional) -* **Output:** `response` (text) - -### Groq Orpheus TTS - -Generate speech using Groq-hosted Orpheus text-to-speech models. - -* **Category:** `audio/generation` -* **Key Inputs:** `text`, `voice`, `speed` -* **Output:** `audio` - ### OpenAI Text-to-Speech - -Generate speech using OpenAI's TTS models. - * **Category:** `audio/generation` -* **Key Inputs:** `text`, `voice` (alloy, echo, etc.), `response_format`, `speed`, `instructions` (optional) +* **Models:** `gpt-4o-mini-tts` (default), `tts-1`, `tts-1-hd` +* **Voices:** 13 voices (`alloy`, `ash`, `ballad`, `cedar`, `coral`, `echo`, `fable`, `marin`, `nova`, `onyx`, `sage`, `shimmer`, `verse`) +* **Key Inputs:** `text`, `voice`, `response_format`, `speed`, `instructions` (natural-language tone/pacing direction on `gpt-4o-mini-tts`) * **Output:** `audio` -### Google Imagen Generator - -Generate images from text using Google's Imagen models. - -* **Category:** `image/generation` -* **Key Inputs:** `number_of_images`, `aspect_ratio`, `image_size`, `guidance_scale`, `negative_prompt` -* **Output:** `images` - -### Google Imagen Edit (Vertex AI only) - -Advanced image editing with inpainting, outpainting, and background swapping. - -* **Category:** `image/edit` -* **Key Inputs:** `image`, `mask`, `project_id`, `location`, `service_account` (JSON content), `edit_mode` -* **Output:** `edited_images` - -### Nano Banana - -Creative image generation using a specialized Gemini model. - -* **Category:** `image/generation` -* **Key Inputs:** `image_1` to `image_5` (optional), `aspect_ratio`, `resolution`, `top_p` -* **Output:** `image` - -### Veo Video Generator (Vertex AI) - -Generate video clips using Google's Veo model on Vertex AI. - -* **Category:** `video/generation` -* **Key Inputs:** `project_id`, `location`, `service_account` (JSON content), `resolution`, `aspect_ratio`, `duration_seconds`, `generate_audio`, `first_frame`/`last_frame` (optional) -* **Output:** `frames`, `audio` - -### Veo Video Generator (Gemini API) - -Generate videos using Veo 2.0 via the Gemini API. Supports text-to-video and image-to-video. - -* **Category:** `video/generation` -* **Key Inputs:** `aspect_ratio`, `duration_seconds`, `negative_prompt`, `image` (optional, for image-to-video) -* **Output:** `frames` - ### ElevenLabs TTS - -Generate speech using ElevenLabs' diverse voices and models. - * **Category:** `audio/generation` -* **Key Inputs:** `text`, `voice_id`, `model_id`, `stability`, `similarity_boost`, `speed`, `style`, `use_speaker_boost` +* **Models:** `eleven_v4` (default), `eleven_v4_turbo`, `eleven_v3`, `eleven_multilingual_v2`, `eleven_flash_v2_5`, `eleven_flash_v2`, `eleven_turbo_v2_5` +* **Key Inputs:** `text`, `voice_id`, `stability`, `similarity_boost`, `speed`, `style`, `use_speaker_boost`, `previous_text` / `next_text` (automatically branches parameters to match v4 requirements) * **Output:** `audio` -### Gemini TTS - -Generate speech using Google's Gemini TTS models. - +### Gemini 3.8 TTS * **Category:** `audio/generation` -* **Key Inputs:** `text`, `voice_id`, `system_prompt` (optional) +* **Models:** `gemini-3.8-flash-tts` (default), `gemini-3.8-flash-lite-tts` +* **Key Inputs:** + * `text`: Supports point-in-time inline tags directly in the transcript (e.g. ``, ``, ``, ``, ``, ``, ``, ``). + * `voice`: Open string targeting any of the 2,000+ Extended Voice Library voices from AI Studio (e.g. `Bodi`, `Lumi`, `Koda`, `Nika`, `Tavi`) or legacy celestial voices (`Puck`, `Zephyr`). + * `style`: Global acting/delivery note passed straight to native `speech_metadata`. * **Output:** `audio` -### Tripo Text-to-3D +### Gemini TTS (Legacy) +* **Category:** `audio/generation` +* **Models:** `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts`, `gemini-3.1-flash-tts-preview` +* **Key Inputs:** Fixed 30 celestial voices dropdown (`Puck`, `Zephyr`, etc.), `pace`, `accent`, `scene`, `audio_profile` prompt-director options. +* **Output:** `audio` -Generate high-quality 3D models from text prompts using Tripo AI's powerful generation engine. +### Nano Banana +* **Category:** `image/generation` +* **Models:** `gemini-nano-banana-2.1` (default), `gemini-3-pro-image`, `gemini-3.1-flash-lite-image`, `gemini-3.1-flash-image`, `gemini-2.5-flash-image` +* **Key Inputs:** `prompt`, `aspect_ratio`, `resolution` (`1K`, `2K`, `4K`), `image_1` to `image_5` (multi-image consistency & editing), `google_search` +* **Output:** `image` +### Veo Video Generator (Gemini API) +* **Category:** `video/generation` +* **Models:** `veo-3.1-generate-preview` (default), `veo-3.1-fast-generate-preview`, `veo-3.1-lite-generate-preview`, `veo-2.0-generate-001` +* **Key Inputs:** `prompt`, `aspect_ratio` (16:9, 9:16), `resolution` (720p, 1080p on 3.1), `duration_seconds` (4 to 8s), `image` (optional, for image-to-video) +* **Output:** `frames` + +### Veo Video Generator (Vertex AI) +* **Category:** `video/generation` +* **Models:** `veo-3.1-generate-001` (default), `veo-3.1-fast-generate-001`, `veo-3.0-generate-001`, `veo-2.0-generate-001` +* **Key Inputs:** `project_id`, `location`, `service_account`, `resolution`, `aspect_ratio`, `duration_seconds`, `generate_audio` (native synchronized audio on 3.x), `first_frame` / `last_frame` +* **Output:** `frames`, `audio` + +### Google Imagen Generator & Edit +* **Category:** `image/generation` & `image/edit` +* **Key Inputs:** Generation controls (`aspect_ratio`, `image_size`, `guidance_scale`) and Vertex AI edit modes (inpainting, outpainting, background swap). +* **Output:** `images` / `edited_images` + +### Groq LLM & Groq Orpheus TTS +* **Category:** `text/generation` & `audio/generation` +* **Key Inputs:** Fast inference using Groq Cloud endpoints for text/vision reasoning and Orpheus voice generation. +* **Output:** `response` (text) / `audio` + +### Tripo Text-to-3D & Image-to-3D * **Category:** `3D/generation` -* **Key Inputs:** `prompt`, `model_version` (v3.0, v2.5, v2.0, v1.4, Turbo), `texture_quality` (standard/detailed), `save_name` -* **Optional Inputs:** `seed`, `face_limit` (500-100000, default 20000), `texture_size` (512/1024/2048), `pbr` (realistic lighting), `negative_prompt` -* **Output:** `glb` (3D model file) - -### Tripo Image-to-3D - -Convert images into detailed 3D models using Tripo AI's advanced image-to-model technology. - -* **Category:** `3D/generation` -* **Key Inputs:** `image`, `model_version` (v3.0, v2.5, v2.0, v1.4), `texture_quality` (standard/detailed), `save_name` -* **Optional Inputs:** `seed`, `texture_seed` (randomness of colors/textures), `face_limit` (500-100000, default 20000), `texture_size` (512/1024/2048), `pbr` (realistic lighting) +* **Key Inputs:** `prompt` or `image`, `model_version` (v3.0, v2.5, v2.0, Turbo), `texture_quality`, `face_limit`, `pbr` * **Output:** `glb` (3D model file) +--- ## Acknowledgements -* The [ComfyUI](https://github.com/comfyanonymous/ComfyUI) team for building the workflow platform that makes these nodes possible. -* [Google](https://deepmind.google/technologies/gemini/) for Gemini, Imagen, and Veo. -* [OpenAI](https://openai.com/) for GPT models and image and speech APIs. -* [Groq](https://groq.com/) for fast inference and hosted text-to-speech support. -* [xAI](https://x.ai/) for Grok image generation and editing models. -* [Black Forest Labs](https://www.blackforestlabs.ai/) and [Replicate](https://replicate.com/) for FLUX model access. -* [ElevenLabs](https://elevenlabs.io/) for speech synthesis. -* [Tripo AI](https://www.tripo3d.ai/) for 3D generation APIs. \ No newline at end of file +* The [ComfyUI](https://github.com/comfyanonymous/ComfyUI) team for building the modular node platform. +* [Google](https://deepmind.google/technologies/gemini/) for Gemini 3.8, Nano Banana, Imagen, and Veo 3.1. +* [OpenAI](https://openai.com/) for GPT-6, GPT Image 2.5, and speech models. +* [ElevenLabs](https://elevenlabs.io/) for Eleven v4 and conversational audio synthesis. +* [Groq](https://groq.com/) for high-throughput LPU inference. +* [Black Forest Labs](https://www.blackforestlabs.ai/) and [Replicate](https://replicate.com/) for FLUX model infrastructure. +* [Tripo AI](https://www.tripo3d.ai/) for native text and image-to-3D APIs. \ No newline at end of file diff --git a/__init__.py b/__init__.py index 1a8f05d..0de0bf2 100644 --- a/__init__.py +++ b/__init__.py @@ -18,13 +18,13 @@ modules = [ "flux2_replicate", "openai_tts", "openai_node", - "sora", "cleanup", "tripoTextToModel", "tripoImageToModel", "grok", "groq_node", "groq_orpheus", + "gemini_tts3", ] NODE_CLASS_MAPPINGS = {} diff --git a/elevenlabs_tts.py b/elevenlabs_tts.py index 2af9c24..2abf2cd 100644 --- a/elevenlabs_tts.py +++ b/elevenlabs_tts.py @@ -12,6 +12,8 @@ class ElevenLabsTTSNode: "required": { "text": ("STRING", {"multiline": True, "default": ""}), "model_id": ([ + "eleven_v4", + "eleven_v4_turbo", "eleven_v3", "eleven_multilingual_v2", "eleven_flash_v2_5", @@ -85,13 +87,19 @@ class ElevenLabsTTSNode: if stability != original_stability: print(f"For 'eleven_v3' model, stability value must be one of: [0.0, 0.5, 1.0] (0.0 = Creative, 0.5 = Natural, 1.0 = Robust). Rounding stability to {stability}.") - voice_settings = { - "stability": stability, - "similarity_boost": similarity_boost, - "speed": speed, - "style": style, - "use_speaker_boost": use_speaker_boost - } + if model_id.startswith("eleven_v4"): + voice_settings = { + "stability": stability, + "similarity_boost": similarity_boost + } + else: + voice_settings = { + "stability": stability, + "similarity_boost": similarity_boost, + "speed": speed, + "style": style, + "use_speaker_boost": use_speaker_boost + } data = { "text": text, @@ -101,9 +109,9 @@ class ElevenLabsTTSNode: "output_format": output_format } - if model_id == "eleven_v3": + if model_id.startswith("eleven_v4") or model_id == "eleven_v3": if previous_text.strip() or next_text.strip(): - print("Providing previous_text or next_text is not yet supported with the 'eleven_v3' model. Ignoring these inputs.") + print(f"Providing previous_text or next_text is not supported with '{model_id}'. Ignoring these inputs.") else: if previous_text.strip(): data["previous_text"] = previous_text @@ -130,7 +138,7 @@ class ElevenLabsTTSNode: @classmethod def IS_CHANGED(cls, **kwargs): - return f"{kwargs.get('text', '')}-{kwargs.get('voice_id', '')}-{kwargs.get('seed', 40)}" + return f"{kwargs.get('text', '')}-{kwargs.get('voice_id', '')}-{kwargs.get('seed', 40)}-{kwargs.get('model_id', '')}" NODE_CLASS_MAPPINGS = {"ElevenLabsTTSNode": ElevenLabsTTSNode} NODE_DISPLAY_NAME_MAPPINGS = {"ElevenLabsTTSNode": "ElevenLabs TTS"} \ No newline at end of file diff --git a/gemini_diarisation.py b/gemini_diarisation.py index a0a54e1..9deab7c 100644 --- a/gemini_diarisation.py +++ b/gemini_diarisation.py @@ -16,18 +16,13 @@ class GeminiDiarisationAPI: "audio": ("AUDIO",), "num_speakers": ("INT", {"default": 2, "min": 1, "max": 10, "step": 1}), "model": ([ + "gemini-3.8-flash", "gemini-3.6-flash", - "gemini-3.5-flash", "gemini-3.5-flash-lite", "gemini-3.1-pro-preview", - "gemini-3.1-flash-lite", - "gemini-3-flash-preview", - "gemini-2.5-flash", - "gemini-2.5-pro", - "gemini-2.5-flash-lite", "gemini-flash-latest", "gemini-flash-lite-latest" - ], {"default": "gemini-3.5-flash"}), + ], {"default": "gemini-3.8-flash"}), "api_key": ("STRING", {"default": "", "multiline": False, "tooltip": "Directly put Gemini API key or .env variable name (GEMINI_API_KEY)"}), "seed": ("INT", {"default": 69, "min": 0, "max": 2147483646, "step": 1}), "temperature": ("FLOAT", {"default": 0.2, "min": 0.0, "max": 2.0, "step": 0.1}) diff --git a/gemini_node.py b/gemini_node.py index 1bfd012..dd5180b 100644 --- a/gemini_node.py +++ b/gemini_node.py @@ -15,18 +15,13 @@ class GeminiChatNode: "prompt": ("STRING", {"multiline": True}), "api_key": ("STRING", {"default": "", "multiline": False, "tooltip": "Directly put Gemini API key or .env variable name (GEMINI_API_KEY)"}), "model": ([ + "gemini-3.8-flash", "gemini-3.6-flash", - "gemini-3.5-flash", "gemini-3.5-flash-lite", "gemini-3.1-pro-preview", - "gemini-3.1-flash-lite", - "gemini-3-flash-preview", - "gemini-2.5-flash", - "gemini-2.5-pro", - "gemini-2.5-flash-lite", "gemini-flash-latest", "gemini-flash-lite-latest" - ], {"default": "gemini-3.5-flash"}), + ], {"default": "gemini-3.8-flash"}), "temperature": ("FLOAT", {"default": 0.2, "min": 0.0, "max": 2.0, "step": 0.1}), "top_p": ("FLOAT", {"default": 0.95, "min": 0.0, "max": 1.0, "step": 0.01}), "thinking": ("BOOLEAN", {"default": False}), diff --git a/gemini_tts.py b/gemini_tts.py index 2f6ca4f..8f8b48f 100644 --- a/gemini_tts.py +++ b/gemini_tts.py @@ -97,4 +97,4 @@ class GeminiTTSNode: return f"{kwargs.get('text', '')}-{kwargs.get('voice_id', '')}-{kwargs.get('temperature', 1.0)}-{kwargs.get('model', '')}-{kwargs.get('seed', 69)}-{kwargs.get('audio_profile', '')}-{kwargs.get('style', '')}-{kwargs.get('pace', '')}-{kwargs.get('accent', '')}-{kwargs.get('scene', '')}" NODE_CLASS_MAPPINGS = {"GeminiTTSNode": GeminiTTSNode} -NODE_DISPLAY_NAME_MAPPINGS = {"GeminiTTSNode": "Gemini TTS"} \ No newline at end of file +NODE_DISPLAY_NAME_MAPPINGS = {"GeminiTTSNode": "Gemini TTS (Legacy)"} \ No newline at end of file diff --git a/gemini_tts3.py b/gemini_tts3.py new file mode 100644 index 0000000..c6cbc10 --- /dev/null +++ b/gemini_tts3.py @@ -0,0 +1,132 @@ +import os +import io +import wave +import torch +from google import genai +from google.genai import types + +class GeminiTTS3Node: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "text": ("STRING", { + "multiline": True, + "default": "", + "tooltip": "Transcript text. Supports inline tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , " + }), + "voice": ("STRING", { + "default": "Bodi", + "multiline": False, + "tooltip": "Voice name from AI Studio (e.g. Bodi, Lumi, Sola, Koda, Nika, Tavi, Kira, Ludo, or legacy celestial names like Puck, Zephyr)" + }), + "model": ([ + "gemini-3.8-flash-tts", + "gemini-3.8-flash-lite-tts", + ], {"default": "gemini-3.8-flash-tts"}), + "api_key": ("STRING", {"default": "", "multiline": False, "tooltip": "Directly put Gemini API key or .env variable name (GEMINI_API_KEY)"}), + "seed": ("INT", {"default": 69, "min": -1, "max": 2147483646, "step": 1}), + "temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.01}), + }, + "optional": { + "style": ("STRING", { + "default": "", + "multiline": True, + "tooltip": "Acting style passed to speech_metadata (e.g. 'Bright, sunny morning greeting with a subtle vocal smile and crisp pacing.')" + }), + } + } + + RETURN_TYPES = ("AUDIO",) + RETURN_NAMES = ("audio",) + FUNCTION = "generate_speech" + CATEGORY = "audio/generation" + + def parse_audio_rate(self, mime_type: str) -> int: + rate = 24000 + if not mime_type: + return rate + for param in mime_type.split(";"): + param = param.strip() + if param.lower().startswith("rate="): + try: + rate = int(param.split("=", 1)[1]) + except (ValueError, IndexError): + pass + return rate + + def generate_speech(self, text, voice, model, api_key, seed, temperature, style=""): + clean_text = text.strip() + if not clean_text: + raise ValueError("Text input cannot be empty.") + + key = os.environ.get(api_key.strip(), api_key.strip()) or os.environ.get("GEMINI_API_KEY") + if not key: + raise ValueError("No API key provided.") + + client = genai.Client(api_key=key) + + transcript = clean_text if clean_text.startswith("## Transcript:") else f"## Transcript:\n{clean_text}" + + metadata = None + if style.strip(): + style_str = style.strip() + if not style_str.lower().startswith("style:"): + style_str = f"Style: {style_str}" + metadata = types.SpeechMetadata(style=style_str) + + part = types.Part(text=transcript, speech_metadata=metadata) if metadata else types.Part(text=transcript) + + config = types.GenerateContentConfig( + temperature=temperature, + seed=seed, + response_modalities=["audio"], + speech_config=types.SpeechConfig( + voice_config=types.VoiceConfig( + prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=voice.strip()) + ) + ) + ) + + audio_data = bytearray() + mime_type = "" + + try: + for chunk in client.models.generate_content_stream( + model=model, + contents=[types.Content(role="user", parts=[part])], + config=config, + ): + if chunk.parts is None: + continue + if chunk.parts[0].inline_data and chunk.parts[0].inline_data.data: + inline_data = chunk.parts[0].inline_data + audio_data.extend(inline_data.data) + mime_type = inline_data.mime_type + except Exception as e: + raise RuntimeError(f"Gemini API Error: {str(e)}") + + if not audio_data: + raise ValueError("API returned a response, but it contained no audio data.") + + sample_rate = self.parse_audio_rate(mime_type) + + if audio_data.startswith(b"RIFF"): + with wave.open(io.BytesIO(audio_data), "rb") as wf: + raw_pcm = wf.readframes(wf.getnframes()) + sample_rate = wf.getframerate() + waveform = torch.frombuffer(bytearray(raw_pcm), dtype=torch.int16) + else: + waveform = torch.frombuffer(bytearray(audio_data), dtype=torch.int16) + + waveform = waveform.to(torch.float32) / 32768.0 + waveform = waveform.unsqueeze(0).unsqueeze(0) + + return ({"waveform": waveform, "sample_rate": sample_rate},) + + @classmethod + def IS_CHANGED(cls, **kwargs): + return f"{kwargs.get('text', '')}-{kwargs.get('voice', '')}-{kwargs.get('model', '')}-{kwargs.get('seed', 69)}-{kwargs.get('temperature', 1.0)}-{kwargs.get('style', '')}" + +NODE_CLASS_MAPPINGS = {"GeminiTTS3Node": GeminiTTS3Node} +NODE_DISPLAY_NAME_MAPPINGS = {"GeminiTTS3Node": "Gemini TTS"} \ No newline at end of file diff --git a/gpt_image_edit.py b/gpt_image_edit.py index 23c26e9..49471fe 100644 --- a/gpt_image_edit.py +++ b/gpt_image_edit.py @@ -12,7 +12,14 @@ class GPTImageNode: def INPUT_TYPES(cls): return { "required": { - "model": (["gpt-image-1", "gpt-image-1-mini", "gpt-image-1.5", "gpt-image-2"],), + "model": ([ + "gpt-image-2.5-sunburst", + "gpt-image-2.5-flare", + "gpt-image-2", + "gpt-image-1.5", + "gpt-image-1", + "gpt-image-1-mini" + ], {"default": "gpt-image-2.5-sunburst"}), "prompt": ("STRING", {"multiline": True}), "api_key": ("STRING", {"multiline": False, "default": "", "tooltip": "Directly put OpenAI API key or .env variable name (OPENAI_API_KEY)"}), "background": (["auto", "transparent", "opaque"], {"default": "auto"}), diff --git a/nano_banana.py b/nano_banana.py index afb568a..fbeff88 100644 --- a/nano_banana.py +++ b/nano_banana.py @@ -14,7 +14,13 @@ class NanoBananaNode: "required": { "prompt": ("STRING", {"multiline": True, "default": ""}), "api_key": ("STRING", {"multiline": False, "default": "", "tooltip": "Directly put Gemini API key or .env variable name (GEMINI_API_KEY)"}), - "model": (["gemini-3-pro-image", "gemini-2.5-flash-image", "gemini-3.1-flash-image", "gemini-3.1-flash-lite-image"]), + "model": ([ + "gemini-nano-banana-2.1", + "gemini-3-pro-image", + "gemini-3.1-flash-lite-image", + "gemini-3.1-flash-image", + "gemini-2.5-flash-image", + ], {"default": "gemini-nano-banana-2.1"}), "aspect_ratio": (["1:1", "2:3", "3:2", "3:4", "4:3", "9:16", "16:9", "21:9"],), "resolution": (["1K", "2K", "4K"], {"default": "1K"}), "temperature": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}), @@ -68,13 +74,13 @@ class NanoBananaNode: tools = None if google_search: - if "gemini-2.5" in model or "gemini-3.1-flash-lite-image" in model: + if "gemini-2.5" in model or "flash-lite" in model: print(f"Ignoring google_search: {model} does not support it.") else: tools = [types.Tool(googleSearch=types.GoogleSearch())] img_config_params = {"aspect_ratio": aspect_ratio} - if "gemini-3-pro" in model: + if "gemini-2.5" not in model and "flash-lite" not in model: img_config_params["image_size"] = resolution config = types.GenerateContentConfig( diff --git a/openai_node.py b/openai_node.py index cd3184b..bfe4811 100644 --- a/openai_node.py +++ b/openai_node.py @@ -13,9 +13,27 @@ class OpenAILLMNode: return { "required": { "prompt": ("STRING", {"multiline": True, "default": ""}), - "model": (["gpt-5.4", "gpt-5.4-pro", "gpt-4.1", "gpt-4.1-mini", "gpt-5", "gpt-5.2", "gpt-5-mini", "gpt-5-nano", "gpt-5.2-pro", "o1", "o3-mini"],), + "model": ([ + "gpt-6", + "gpt-6.1-sol", + "gpt-6-luna", + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-5.5", + "gpt-5.5-pro", + "gpt-5.4", + "gpt-5.4-pro", + "gpt-5.2", + "gpt-5.2-pro", + "gpt-5", + "gpt-5-mini", + "gpt-5-nano", + "gpt-4.1", + "gpt-4.1-mini", + ], {"default": "gpt-6"}), "temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.01}), - "reasoning_effort": (["low", "medium", "high"],), + "reasoning_effort": (["none", "low", "medium", "high"], {"default": "medium"}), "api_key": ("STRING", {"multiline": False, "default": "", "tooltip": "Directly put OpenAI API key or .env variable name (OPENAI_API_KEY)"}), "max_output_tokens": ("INT", {"default": 16384, "min": 1, "max": 32768, "step": 1}) }, @@ -61,8 +79,10 @@ class OpenAILLMNode: "max_output_tokens": max_output_tokens } + # Pass reasoning effort only if supported and not set to "none" if not model.startswith("gpt-4"): - request_params["reasoning"] = {"effort": reasoning_effort} + if reasoning_effort != "none": + request_params["reasoning"] = {"effort": reasoning_effort} else: print(f"Skipping reasoning parameter for {model} (not supported)") diff --git a/openai_tts.py b/openai_tts.py index 96c4864..e6c89af 100644 --- a/openai_tts.py +++ b/openai_tts.py @@ -15,18 +15,22 @@ class OpenAITTSNode: "gpt-4o-mini-tts", "tts-1", "tts-1-hd" - ],), + ], {"default": "gpt-4o-mini-tts"}), "voice": ([ "alloy", "ash", + "ballad", + "cedar", "coral", "echo", "fable", - "onyx", + "marin", "nova", + "onyx", "sage", - "shimmer" - ],), + "shimmer", + "verse" + ], {"default": "alloy"}), "response_format": ([ "mp3", "opus", @@ -34,12 +38,16 @@ class OpenAITTSNode: "flac", "wav", "pcm" - ],), + ], {"default": "mp3"}), "speed": ("FLOAT", {"default": 1.0, "min": 0.25, "max": 4.0, "step": 0.01}), "api_key": ("STRING", {"multiline": False, "default": "", "tooltip": "Directly put OpenAI API key or .env variable name (OPENAI_API_KEY)"}), }, "optional": { - "instructions": ("STRING", {"multiline": True, "default": ""}), + "instructions": ("STRING", { + "multiline": True, + "default": "", + "tooltip": "Supported on gpt-4o-mini-tts. Natural-language guidance for tone, emotion, or pacing (e.g. 'Whisper excitedly', 'Speak like a calm news anchor')." + }), } } @@ -78,7 +86,7 @@ class OpenAITTSNode: # Add instructions only if provided and model supports it if instructions.strip() and model not in ["tts-1", "tts-1-hd"]: - data["instructions"] = instructions + data["instructions"] = instructions.strip() response = requests.post(url, json=data, headers=headers) @@ -100,7 +108,7 @@ class OpenAITTSNode: @classmethod def IS_CHANGED(cls, **kwargs): - return f"{kwargs.get('text', '')}-{kwargs.get('voice', '')}-{kwargs.get('model', '')}" + return f"{kwargs.get('text', '')}-{kwargs.get('voice', '')}-{kwargs.get('model', '')}-{kwargs.get('speed', 1.0)}-{kwargs.get('instructions', '')}" NODE_CLASS_MAPPINGS = {"OpenAITTSNode": OpenAITTSNode} NODE_DISPLAY_NAME_MAPPINGS = {"OpenAITTSNode": "OpenAI Text-to-Speech"} \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 91caf23..a26e9b9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "externalapi-helpers" description = "ComfyUI nodes for FLUX.1 Kontext, FLUX.2, Gemini, Veo, Imagen, OpenAI GPT, GPT-Image, Sora, xAI Grok, Groq, ElevenLabs, Nano Banana, and Tripo" -version = "1.1.7" +version = "1.1.8" license = {file = "LICENSE"} # classifiers = [ # # For OS-independent nodes (works on all operating systems) diff --git a/sora.py b/sora.py deleted file mode 100644 index fa4679e..0000000 --- a/sora.py +++ /dev/null @@ -1,90 +0,0 @@ -import time -import io -import torch -import numpy as np -import av -from PIL import Image -import os -from openai import OpenAI - -class SoraGen: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "prompt": ("STRING", {"multiline": True, "default": "A calico cat playing a piano on stage"}), - "api_key": ("STRING", {"multiline": False, "default": "", "tooltip": "Directly put OpenAI API key or .env variable name (OPENAI_API_KEY)"}), - "model": (["sora-2", "sora-2-pro"], {"default": "sora-2"}), - "size": (["720x1280", "1280x720", "1024x1792", "1792x1024"], {"default": "1280x720"}), - "duration": (["4", "8", "12"], {"default": "4"}), - "seed": ("INT", {"default": 69, "min": 1, "max": 2147483646, "step": 1}), - }, - "optional": {"input_image": ("IMAGE",)} - } - - RETURN_TYPES = ("IMAGE", "AUDIO") - RETURN_NAMES = ("frames", "audio") - FUNCTION = "generate_video" - CATEGORY = "video/generation" - OUTPUT_IS_LIST = (True, False) - - def generate_video(self, prompt, api_key, model, size, duration, seed, input_image=None): - key = os.environ.get(api_key.strip(), api_key.strip()) or os.environ.get("OPENAI_API_KEY") - if not key: raise ValueError("No API key provided.") - client = OpenAI(api_key=key) - api_args = {"prompt": prompt, "model": model, "size": size, "seconds": duration} - - img_buf = None - if input_image is not None: - img_buf = io.BytesIO() - # Convert Tensor (Batch, H, W, C) -> Numpy (H, W, C) -> PIL - arr = (input_image.cpu().numpy()[0] * 255).astype(np.uint8) - Image.fromarray(arr).save(img_buf, format="JPEG", quality=95) - img_buf.seek(0) - api_args["input_reference"] = ("ref.jpg", img_buf, "image/jpeg") - - try: - job = client.videos.create(**api_args) - print(f"Job started: {job.id}") - - while (status := client.videos.retrieve(job.id)).status not in ["completed", "failed"]: - time.sleep(4) - - if status.status == "failed": - raise Exception(f"API Error: {status.error.message}") - - video_bytes = io.BytesIO(client.videos.download_content(video_id=job.id).read()) - - container = av.open(video_bytes) - frames = [] - for frame in container.decode(video=0): - img = frame.to_rgb().to_ndarray().astype(np.float32) / 255.0 - frames.append(torch.from_numpy(img).unsqueeze(0)) - container.close() - - video_bytes.seek(0) - container = av.open(video_bytes) - audio = None - - if container.streams.audio: - audio_data = [f.to_ndarray() for f in container.decode(audio=0)] - if audio_data: - waveform = torch.from_numpy(np.concatenate(audio_data, axis=1)).float() - # Normalize Audio - if audio_data[0].dtype == np.int16: waveform /= 32768.0 - elif audio_data[0].dtype == np.int32: waveform /= 2147483648.0 - - audio = { - "waveform": waveform.unsqueeze(0), - "sample_rate": container.streams.audio[0].rate - } - container.close() - - return ([torch.cat(frames, dim=0)], audio) - - finally: - if img_buf: - img_buf.close() - -NODE_CLASS_MAPPINGS = {"SoraGen": SoraGen} -NODE_DISPLAY_NAME_MAPPINGS = {"SoraGen": "Sora 2 (OpenAI)"} \ No newline at end of file diff --git a/veo.py b/veo.py index fdfb123..ee069e1 100644 --- a/veo.py +++ b/veo.py @@ -31,15 +31,19 @@ class VeoVertexVideoGenerator: ], {"default": "us-central1"}), "service_account": ("STRING", {"multiline": True, "default": "", "tooltip": "Paste service account JSON content"}), "model": ([ - "veo-2.0-generate-001", "veo-2.0-generate-exp", "veo-2.0-generate-preview", - "veo-3.0-generate-001", "veo-3.0-fast-generate-001", - "veo-3.1-generate-001", "veo-3.1-fast-generate-001" - ], {"default": "veo-3.0-generate-001"}), + "veo-3.1-generate-001", + "veo-3.1-fast-generate-001", + "veo-3.0-generate-001", + "veo-3.0-fast-generate-001", + "veo-2.0-generate-001", + "veo-2.0-generate-exp", + "veo-2.0-generate-preview", + ], {"default": "veo-3.1-generate-001"}), "resolution": (["720p", "1080p"], {"default": "720p"}), "aspect_ratio": (["16:9", "9:16"], {"default": "16:9"}), - "duration_seconds": ("INT", {"default": 4, "min": 4, "max": 8, "step": 1}), + "duration_seconds": ("INT", {"default": 8, "min": 4, "max": 8, "step": 1}), "seed": ("INT", {"default": 69, "min": 1, "max": 2147483646, "step": 1}), - "generate_audio": ("BOOLEAN", {"default": False}), + "generate_audio": ("BOOLEAN", {"default": True}), "fps": (["24"], {"default": "24"}), }, "optional": { @@ -66,16 +70,29 @@ class VeoVertexVideoGenerator: try: client = genai.Client(vertexai=True, project=project_id, location=location) + + # Veo 2.0 requires minimum 5s duration and does not support audio generation + dur = duration_seconds + is_veo2 = "veo-2.0" in model + if is_veo2 and dur < 5: + dur = 5 + + should_gen_audio = False if is_veo2 else generate_audio - config = types.GenerateVideosConfig( - resolution=resolution, - aspect_ratio=aspect_ratio, - duration_seconds=duration_seconds, - generate_audio=generate_audio, - fps=int(fps), - seed=seed if seed != -1 else None, - negative_prompt=negative_prompt.strip() if negative_prompt else None - ) + config_kwargs = { + "aspect_ratio": aspect_ratio, + "duration_seconds": dur, + "fps": int(fps), + "seed": seed if seed != -1 else None, + "negative_prompt": negative_prompt.strip() if negative_prompt else None + } + + # Only Veo 3.x accepts resolution and audio generation flags + if not is_veo2: + config_kwargs["resolution"] = resolution + config_kwargs["generate_audio"] = should_gen_audio + + config = types.GenerateVideosConfig(**config_kwargs) # Helper: Tensor -> Bytes def tensor_to_bytes(t): @@ -112,7 +129,7 @@ class VeoVertexVideoGenerator: # Decode Audio audio = None - if generate_audio: + if should_gen_audio: video_bytes.seek(0) container = av.open(video_bytes) if container.streams.audio: diff --git a/veo_api.py b/veo_api.py index fb469c7..89eb064 100644 --- a/veo_api.py +++ b/veo_api.py @@ -14,9 +14,15 @@ class VeoGeminiVideoGenerator: return { "required": { "prompt": ("STRING", {"multiline": True, "default": "a cat reading a book"}), - "model": (["veo-2.0-generate-001"], {"default": "veo-2.0-generate-001"}), + "model": ([ + "veo-3.1-generate-preview", + "veo-3.1-fast-generate-preview", + "veo-3.1-lite", + "veo-2.0-generate-001", + ], {"default": "veo-3.1-generate-preview"}), "aspect_ratio": (["16:9", "9:16"], {"default": "16:9"}), - "duration_seconds": ("INT", {"default": 8, "min": 5, "max": 8, "step": 1}), + "resolution": (["720p", "1080p"], {"default": "720p"}), + "duration_seconds": ("INT", {"default": 8, "min": 4, "max": 8, "step": 1}), "api_key": ("STRING", {"default": "", "multiline": False, "tooltip": "Directly put Gemini API key or .env variable name (GEMINI_API_KEY)"}), "seed": ("INT", {"default": 69, "min": -1, "max": 2147483646, "step": 1}), }, @@ -32,7 +38,7 @@ class VeoGeminiVideoGenerator: CATEGORY = "video/generation" OUTPUT_IS_LIST = (True,) - def generate_video(self, seed, prompt, model, aspect_ratio, duration_seconds, api_key, + def generate_video(self, seed, prompt, model, aspect_ratio, resolution, duration_seconds, api_key, negative_prompt=None, image=None): # 1. Setup Client @@ -40,24 +46,35 @@ class VeoGeminiVideoGenerator: if not key: raise ValueError("API Key required") client = genai.Client(http_options={"api_version": "v1beta"}, api_key=key) - # 2. Prepare Config (Seed used for ComfyUI caching only) + # 2. Duration compatibility clamp (Veo 2.0 requires minimum 5s) + dur = duration_seconds + if "veo-2.0" in model and dur < 5: + dur = 5 + + # 3. Prepare Config + config_params = { + "aspect_ratio": aspect_ratio, + "duration_seconds": dur, + "negative_prompt": negative_prompt.strip() if negative_prompt else None, + } + + # Resolution is supported on Veo 3.x + if "veo-3" in model: + config_params["resolution"] = resolution + gen_kwargs = { "model": model, "prompt": prompt, - "config": types.GenerateVideosConfig( - aspect_ratio=aspect_ratio, - duration_seconds=duration_seconds, - negative_prompt=negative_prompt.strip() if negative_prompt else None, - ) + "config": types.GenerateVideosConfig(**config_params) } - # 3. Handle Image (In-Memory, ComfyUI Tensors are always [B,H,W,C]) + # 4. Handle Image Input (Image-to-Video) if image is not None: buf = io.BytesIO() Image.fromarray((image[0].cpu().numpy() * 255).astype(np.uint8)).save(buf, format="PNG") gen_kwargs["image"] = types.Image(image_bytes=buf.getvalue(), mime_type="image/png") - # 4. Generate & Poll + # 5. Generate & Poll op = client.models.generate_videos(**gen_kwargs) print(f"Gemini Veo Operation: {op.name}") @@ -68,7 +85,7 @@ class VeoGeminiVideoGenerator: if not op.response or not op.response.generated_videos: raise Exception("No videos generated") - # 5. Download & Decode (Direct Memory Stream) + # 6. Download & Decode Video Frames video_bytes = io.BytesIO(client.files.download(file=op.response.generated_videos[0].video)) container = av.open(video_bytes)