diff --git a/README.md b/README.md index fa0412f..ff3c5a1 100644 --- a/README.md +++ b/README.md @@ -2,15 +2,16 @@ A collection of powerful custom nodes for ComfyUI that connect your local workflows to closed-source AI models via their APIs. Use Google's Gemini, Imagen, Veo, OpenAI's GPT-Image-1, and Black Forest Labs' FLUX models directly within ComfyUI. - - ## Key Features * **FLUX Kontext Pro & Max:** Image-to-image transformations using the FLUX models via the Replicate API. -* **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models. +* **Flux.2 (Replicate):** Generate images using the latest FLUX.2 models (Pro, Max, Dev) via Replicate. +* **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models. Now supports audio input. * **Gemini Segmentation:** Generate segmentation masks for objects in an image using Gemini. * **Gemini Speaker Diarization:** Separate audio into different speaker tracks using Gemini. * **GPT Image Edit:** OpenAI's `gpt-image-1` for prompt-based image editing and inpainting. Simply mask an area and describe the change you want to see. +* **OpenAI LLM:** Access OpenAI's powerful language models (GPT-4, GPT-5, o1, etc.) for text generation and reasoning. +* **OpenAI Text-to-Speech:** Generate high-quality speech using OpenAI's TTS models. * **Google Imagen Generator & Edit:** Create and edit images with Google's Imagen models, with support for Vertex AI. * **Nano Banana:** A creative image generation node using a specialized Gemini model. * **Veo Video Generator:** Generate high-quality video clips from text prompts using Google's Veo model via Vertex AI or the Gemini API. @@ -46,7 +47,7 @@ All nodes in this collection require API keys to function. * **FLUX Nodes (Replicate):** You will need a [Replicate API Token](https://replicate.com/account/api-tokens). * **Gemini, Imagen, Nano Banana, Gemini TTS, Gemini Diarization, and Veo (Gemini API) Nodes:** You will need a [Google AI Studio API Key](https://aistudio.google.com/app/api-keys). -* **GPT Image Edit Node:** You will need an [OpenAI API Key](https://platform.openai.com/api-keys). +* **OpenAI Nodes (GPT Image Edit, OpenAI LLM, OpenAI TTS):** You will need an [OpenAI API Key](https://platform.openai.com/api-keys). * **ElevenLabs TTS Node:** You will need an [ElevenLabs API Key](https://elevenlabs.io/). * **Vertex AI Nodes (Imagen Edit, Veo Vertex AI):** You will need a Google Cloud Project ID, a service account with appropriate permissions, and the location for the resources. @@ -71,19 +72,38 @@ These nodes allow you to transform an input image based on a text prompt. They a * **Output:** * `image`: The generated image. +### Flux.2 (Replicate) + +Generate images using Black Forest Labs' FLUX.2 models via the Replicate API. + +* **Category:** `image/generation` +* **Inputs:** + * `prompt`: The text prompt for image generation. + * `api_key`: Your Replicate API token. + * `model`: Choose between `flux-2-max`, `flux-2-pro`, or `flux-2-dev`. + * `aspect_ratio`: The desired aspect ratio for the generated image. + * `output_format`: `webp`, `jpg`, or `png`. + * `output_quality`: Quality of the output image (0-100). + * `image_1` to `image_5` (Optional): Input images for image-to-image or control tasks. +* **Output:** + * `image`: The generated image. + ### Gemini Chat -A versatile node for text generation and image analysis. Use it to understand an image's content or to generate creative text for other nodes. +A versatile node for text generation and image/audio analysis. Use it to understand an image's content, analyze audio, or to generate creative text for other nodes. * **Category:** `text/generation` * **Inputs:** * `prompt`: The text prompt or question you want to ask the model. - * `image` (Optional): An input image for the model to analyze. + * `model`: The Gemini model to use (e.g., `gemini-2.5-pro`, `gemini-2.5-flash`). + * `temperature`: Controls the creativity of the output. + * `thinking`: Enables the model's thinking/reasoning process. + * `seed`: Seed for reproducibility. * `api_key`: Your API key from Google AI Studio. - * `model`: The Gemini model to use (e.g., `gemini-2.5-pro`). * `system_instruction` (Optional): Provide context or rules for how the model should behave. - * `temperature`: Controls the creativity of the output. Higher is more creative. - * `thinking`: Enables the model's thinking/reasoning process (Gemini 2.5 Pro). + * `thinking_budget` (Optional): Token budget for thinking. + * `image` (Optional): An input image for the model to analyze. + * `audio` (Optional): An input audio for the model to analyze. * **Output:** * `response`: The text generated by the Gemini model. @@ -95,9 +115,12 @@ This node uses a Gemini model to generate segmentation masks for specified objec * **Inputs:** * `image`: The source image for segmentation. * `segment_prompt`: A text description of the objects to segment (e.g., "the car", "all people"). - * `api_key`: Your API key from Google AI Studio. * `model`: The Gemini model to use. - * `...other_params`: Controls for temperature, thinking, and seed. + * `temperature`: Controls randomness. + * `thinking`: Enable thinking process. + * `seed`: Seed for reproducibility. + * `api_key`: Your API key from Google AI Studio. + * `thinking_budget` (Optional): Token budget for thinking. * **Output:** * `mask`: A black and white mask of the segmented objects. @@ -108,12 +131,13 @@ Separate audio into different speaker tracks using Gemini. * **Category:** `audio/diarise` * **Inputs:** * `audio`: The input audio to process. - * `num_speakers`: The expected number of speakers (default: 2). - * `model`: The Gemini model to use (e.g., `gemini-2.5-flash`). + * `num_speakers`: The expected number of speakers. + * `model`: The Gemini model to use. * `api_key`: Your API key from Google AI Studio. - * `thinking`: Enable thinking process (optional). - * `thinking_budget`: Token budget for thinking (optional). - * `...other_params`: Controls for temperature and seed. + * `seed`: Seed for reproducibility. + * `temperature`: Controls randomness. + * `thinking` (Optional): Enable thinking process. + * `thinking_budget` (Optional): Token budget for thinking. * **Output:** * `speaker_1` to `speaker_4`: Audio tracks for up to 4 separated speakers. @@ -125,13 +149,44 @@ This node uses OpenAI's API to perform powerful, prompt-based inpainting and edi * **Inputs:** * `image`: The source image to edit. * `mask` (Optional): A black and white mask. The model will edit the **white area** of the mask. - * `prompt`: A description of the edit to perform (e.g., "Add a small red boat on the water", "Remove the person on the left"). + * `prompt`: A description of the edit to perform. * `api_key`: Your API key from OpenAI. * `...other_params`: Various quality and formatting options for the OpenAI API. * **Output:** * `image`: The edited image. -**Note:** If a mask is provided, the edits will be constrained to the masked region. If no mask is provided, the model will attempt to edit the entire image based on the prompt. +### OpenAI LLM + +Access OpenAI's powerful language models for text generation and reasoning. + +* **Category:** `text/generation` +* **Inputs:** + * `prompt`: The text prompt. + * `model`: The OpenAI model to use (e.g., `gpt-4.1`, `o1`, `gpt-5`). + * `temperature`: Controls randomness. + * `reasoning_effort`: Effort level for reasoning models. + * `api_key`: Your OpenAI API key. + * `max_output_tokens`: Maximum number of tokens to generate. + * `system_instruction` (Optional): System level instructions. + * `image` (Optional): Input image for multimodal models. +* **Output:** + * `response`: The generated text response. + +### OpenAI Text-to-Speech + +Generate high-quality speech from text using OpenAI's TTS models. + +* **Category:** `audio/generation` +* **Inputs:** + * `text`: The text to convert to speech. + * `model`: The TTS model to use (e.g., `gpt-4o-mini-tts`, `tts-1`). + * `voice`: The voice to use (e.g., `alloy`, `echo`). + * `response_format`: Output audio format. + * `speed`: Speaking speed. + * `api_key`: Your OpenAI API key. + * `instructions` (Optional): Instructions for the model (supported by some models). +* **Output:** + * `audio`: The generated audio. ### Google Imagen Generator @@ -206,8 +261,6 @@ Generate videos using Google's Veo 2.0 model via the Gemini API. Supports text-t * **Output:** * `frames`: The generated video frames. ---- - ### ElevenLabs TTS Generate speech from text using the ElevenLabs API. @@ -249,4 +302,4 @@ Generate speech from text using Google's Gemini TTS models. * The [ComfyUI](https://github.com/comfyanonymous/ComfyUI) team for creating such a flexible and powerful platform. * [Google](https://deepmind.google/technologies/gemini/), [OpenAI](https://openai.com/), and [Black Forest Labs](https://www.blackforestlabs.ai/) for developing these incredible models. -* [Replicate](https://replicate.com/) for providing easy API access to a wide range of models. +* [Replicate](https://replicate.com/) for providing easy API access to a wide range of models. \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 8ebb8f0..075acb2 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "externalapi-helpers" description = "Various ComfyUI nodes for Gemini, Replicate and OpenAI" -version = "1.0.4" +version = "1.0.5" license = {file = "LICENSE"} # classifiers = [ # # For OS-independent nodes (works on all operating systems) diff --git a/requirements.txt b/requirements.txt index a9132eb..41a8817 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,4 +4,5 @@ numpy torch google-genai opencv-python -soundfile \ No newline at end of file +soundfile +openai \ No newline at end of file