diff --git a/README.md b/README.md index aef92ba..fa0412f 100644 --- a/README.md +++ b/README.md @@ -9,14 +9,13 @@ A collection of powerful custom nodes for ComfyUI that connect your local workfl * **FLUX Kontext Pro & Max:** Image-to-image transformations using the FLUX models via the Replicate API. * **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models. * **Gemini Segmentation:** Generate segmentation masks for objects in an image using Gemini. +* **Gemini Speaker Diarization:** Separate audio into different speaker tracks using Gemini. * **GPT Image Edit:** OpenAI's `gpt-image-1` for prompt-based image editing and inpainting. Simply mask an area and describe the change you want to see. * **Google Imagen Generator & Edit:** Create and edit images with Google's Imagen models, with support for Vertex AI. * **Nano Banana:** A creative image generation node using a specialized Gemini model. -* **Veo Text-to-Video:** Generate high-quality video clips from text prompts using Google's Veo model via Vertex AI. +* **Veo Video Generator:** Generate high-quality video clips from text prompts using Google's Veo model via Vertex AI or the Gemini API. * **ElevenLabs TTS:** Generate high-quality speech from text using ElevenLabs' diverse range of voices and models. * **Gemini TTS:** Create speech from text using Google's Gemini models. -* **Seamless Integration:** All nodes are designed to work seamlessly with standard ComfyUI inputs (IMAGE, MASK, STRING) and outputs, allowing you to chain them into complex and creative workflows. -* **Secure & Simple:** Simply provide your API key in the node's input field to get started. --- @@ -46,10 +45,10 @@ A collection of powerful custom nodes for ComfyUI that connect your local workfl All nodes in this collection require API keys to function. * **FLUX Nodes (Replicate):** You will need a [Replicate API Token](https://replicate.com/account/api-tokens). -* **Gemini, Imagen, Nano Banana, and Gemini TTS Nodes:** You will need a [Google AI Studio API Key](https://aistudio.google.com/app/api-keys). +* **Gemini, Imagen, Nano Banana, Gemini TTS, Gemini Diarization, and Veo (Gemini API) Nodes:** You will need a [Google AI Studio API Key](https://aistudio.google.com/app/api-keys). * **GPT Image Edit Node:** You will need an [OpenAI API Key](https://platform.openai.com/api-keys). * **ElevenLabs TTS Node:** You will need an [ElevenLabs API Key](https://elevenlabs.io/). -* **Vertex AI Nodes (Imagen Edit, Veo):** You will need a Google Cloud Project ID, a service account with appropriate permissions, and the location for the resources. +* **Vertex AI Nodes (Imagen Edit, Veo Vertex AI):** You will need a Google Cloud Project ID, a service account with appropriate permissions, and the location for the resources. You can paste your key directly into the `api_key` field on the corresponding node. For Vertex AI nodes, you will need to provide the project ID, location, and path to your service account JSON file. @@ -102,6 +101,22 @@ This node uses a Gemini model to generate segmentation masks for specified objec * **Output:** * `mask`: A black and white mask of the segmented objects. +### Gemini Speaker Diarization + +Separate audio into different speaker tracks using Gemini. + +* **Category:** `audio/diarise` +* **Inputs:** + * `audio`: The input audio to process. + * `num_speakers`: The expected number of speakers (default: 2). + * `model`: The Gemini model to use (e.g., `gemini-2.5-flash`). + * `api_key`: Your API key from Google AI Studio. + * `thinking`: Enable thinking process (optional). + * `thinking_budget`: Token budget for thinking (optional). + * `...other_params`: Controls for temperature and seed. +* **Output:** + * `speaker_1` to `speaker_4`: Audio tracks for up to 4 separated speakers. + ### GPT Image Edit This node uses OpenAI's API to perform powerful, prompt-based inpainting and editing. @@ -161,7 +176,7 @@ A creative image generation node that can take a combination of text and up to f * **Output:** * `image`: The generated image. -### Veo Text-to-Video (Vertex AI) +### Veo Video Generator (Vertex AI) Generate short, high-quality video clips from a text description using Google's Veo model on Vertex AI. @@ -175,6 +190,22 @@ Generate short, high-quality video clips from a text description using Google's * **Output:** * `frames`: The generated video frames, output as an image batch. +### Veo Video Generator (Gemini API) + +Generate videos using Google's Veo 2.0 model via the Gemini API. Supports text-to-video and image-to-video. + +* **Category:** `video/generation` +* **Inputs:** + * `prompt`: A text description of the video. + * `image` (Optional): An input image for image-to-video generation. + * `api_key`: Your API key from Google AI Studio. + * `model`: The Veo model to use (e.g., `veo-2.0-generate-001`). + * `aspect_ratio`: Desired aspect ratio (16:9 or 9:16). + * `duration_seconds`: Duration of the video (e.g., 5-8 seconds). + * `...other_params`: Controls for negative prompt and seed. +* **Output:** + * `frames`: The generated video frames. + --- ### ElevenLabs TTS diff --git a/imagen.py b/imagen.py index 95eb46c..1a7141a 100644 --- a/imagen.py +++ b/imagen.py @@ -17,6 +17,11 @@ class GoogleImagenNode: "number_of_images": ("INT", {"default": 1, "min": 1, "max": 4, "step": 1}), "aspect_ratio": (["1:1", "9:16", "16:9", "4:3", "3:4"],), "image_size": (["1K", "2K"],), + "seed": ("INT", {"default": 69, "min": 1, "max": 2147483646, "step": 1}), + "guidance_scale": ("FLOAT", {"default": 7.5, "min": 1.0, "max": 20.0, "step": 0.1}), + }, + "optional": { + "negative_prompt": ("STRING", {"multiline": True, "default": ""}), } } @@ -39,7 +44,7 @@ class GoogleImagenNode: return torch.stack(tensors) - def generate_images(self, prompt, api_key, model, number_of_images, aspect_ratio, image_size): + def generate_images(self, prompt, api_key, model, number_of_images, aspect_ratio, image_size, seed, guidance_scale, negative_prompt=""): try: key = api_key.strip() or os.environ.get("GEMINI_API_KEY") if not key: @@ -47,12 +52,18 @@ class GoogleImagenNode: client = genai.Client(api_key=key) - config = types.GenerateImagesConfig( - number_of_images=number_of_images, - include_rai_reason=True, - output_mime_type="image/jpeg", - aspect_ratio=aspect_ratio, - ) + config_params = { + "number_of_images": number_of_images, + "aspect_ratio": aspect_ratio, + "add_watermark": False, + "seed": seed, + "guidance_scale": guidance_scale, + } + + if negative_prompt and negative_prompt.strip(): + config_params["negative_prompt"] = negative_prompt + + config = types.GenerateImagesConfig(**config_params) if model in ["models/imagen-4.0-ultra-generate-001", "models/imagen-4.0-generate-001"]: config.image_size = image_size diff --git a/pyproject.toml b/pyproject.toml index 537a1b7..4f3fc16 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "externalapi-helpers" description = "Various ComfyUI nodes for Gemini, Replicate and OpenAI" -version = "1.0.1" +version = "1.0.2" license = {file = "LICENSE"} # classifiers = [ # # For OS-independent nodes (works on all operating systems)