Updated README, req
This commit is contained in:
Aryan
@@ -2,15 +2,16 @@
|
||||
|
||||
A collection of powerful custom nodes for ComfyUI that connect your local workflows to closed-source AI models via their APIs. Use Google's Gemini, Imagen, Veo, OpenAI's GPT-Image-1, and Black Forest Labs' FLUX models directly within ComfyUI.
|
||||
|
||||
|
||||
|
||||
## Key Features
|
||||
|
||||
* **FLUX Kontext Pro & Max:** Image-to-image transformations using the FLUX models via the Replicate API.
|
||||
* **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models.
|
||||
* **Flux.2 (Replicate):** Generate images using the latest FLUX.2 models (Pro, Max, Dev) via Replicate.
|
||||
* **Gemini Chat:** Google's powerful multimodal AI. Ask questions about an image, generate detailed descriptions or create prompts for other models. Supports thinking budget controls for applicable models. Now supports audio input.
|
||||
* **Gemini Segmentation:** Generate segmentation masks for objects in an image using Gemini.
|
||||
* **Gemini Speaker Diarization:** Separate audio into different speaker tracks using Gemini.
|
||||
* **GPT Image Edit:** OpenAI's `gpt-image-1` for prompt-based image editing and inpainting. Simply mask an area and describe the change you want to see.
|
||||
* **OpenAI LLM:** Access OpenAI's powerful language models (GPT-4, GPT-5, o1, etc.) for text generation and reasoning.
|
||||
* **OpenAI Text-to-Speech:** Generate high-quality speech using OpenAI's TTS models.
|
||||
* **Google Imagen Generator & Edit:** Create and edit images with Google's Imagen models, with support for Vertex AI.
|
||||
* **Nano Banana:** A creative image generation node using a specialized Gemini model.
|
||||
* **Veo Video Generator:** Generate high-quality video clips from text prompts using Google's Veo model via Vertex AI or the Gemini API.
|
||||
@@ -46,7 +47,7 @@ All nodes in this collection require API keys to function.
|
||||
|
||||
* **FLUX Nodes (Replicate):** You will need a [Replicate API Token](https://replicate.com/account/api-tokens).
|
||||
* **Gemini, Imagen, Nano Banana, Gemini TTS, Gemini Diarization, and Veo (Gemini API) Nodes:** You will need a [Google AI Studio API Key](https://aistudio.google.com/app/api-keys).
|
||||
* **GPT Image Edit Node:** You will need an [OpenAI API Key](https://platform.openai.com/api-keys).
|
||||
* **OpenAI Nodes (GPT Image Edit, OpenAI LLM, OpenAI TTS):** You will need an [OpenAI API Key](https://platform.openai.com/api-keys).
|
||||
* **ElevenLabs TTS Node:** You will need an [ElevenLabs API Key](https://elevenlabs.io/).
|
||||
* **Vertex AI Nodes (Imagen Edit, Veo Vertex AI):** You will need a Google Cloud Project ID, a service account with appropriate permissions, and the location for the resources.
|
||||
|
||||
@@ -71,19 +72,38 @@ These nodes allow you to transform an input image based on a text prompt. They a
|
||||
* **Output:**
|
||||
* `image`: The generated image.
|
||||
|
||||
### Flux.2 (Replicate)
|
||||
|
||||
Generate images using Black Forest Labs' FLUX.2 models via the Replicate API.
|
||||
|
||||
* **Category:** `image/generation`
|
||||
* **Inputs:**
|
||||
* `prompt`: The text prompt for image generation.
|
||||
* `api_key`: Your Replicate API token.
|
||||
* `model`: Choose between `flux-2-max`, `flux-2-pro`, or `flux-2-dev`.
|
||||
* `aspect_ratio`: The desired aspect ratio for the generated image.
|
||||
* `output_format`: `webp`, `jpg`, or `png`.
|
||||
* `output_quality`: Quality of the output image (0-100).
|
||||
* `image_1` to `image_5` (Optional): Input images for image-to-image or control tasks.
|
||||
* **Output:**
|
||||
* `image`: The generated image.
|
||||
|
||||
### Gemini Chat
|
||||
|
||||
A versatile node for text generation and image analysis. Use it to understand an image's content or to generate creative text for other nodes.
|
||||
A versatile node for text generation and image/audio analysis. Use it to understand an image's content, analyze audio, or to generate creative text for other nodes.
|
||||
|
||||
* **Category:** `text/generation`
|
||||
* **Inputs:**
|
||||
* `prompt`: The text prompt or question you want to ask the model.
|
||||
* `image` (Optional): An input image for the model to analyze.
|
||||
* `model`: The Gemini model to use (e.g., `gemini-2.5-pro`, `gemini-2.5-flash`).
|
||||
* `temperature`: Controls the creativity of the output.
|
||||
* `thinking`: Enables the model's thinking/reasoning process.
|
||||
* `seed`: Seed for reproducibility.
|
||||
* `api_key`: Your API key from Google AI Studio.
|
||||
* `model`: The Gemini model to use (e.g., `gemini-2.5-pro`).
|
||||
* `system_instruction` (Optional): Provide context or rules for how the model should behave.
|
||||
* `temperature`: Controls the creativity of the output. Higher is more creative.
|
||||
* `thinking`: Enables the model's thinking/reasoning process (Gemini 2.5 Pro).
|
||||
* `thinking_budget` (Optional): Token budget for thinking.
|
||||
* `image` (Optional): An input image for the model to analyze.
|
||||
* `audio` (Optional): An input audio for the model to analyze.
|
||||
* **Output:**
|
||||
* `response`: The text generated by the Gemini model.
|
||||
|
||||
@@ -95,9 +115,12 @@ This node uses a Gemini model to generate segmentation masks for specified objec
|
||||
* **Inputs:**
|
||||
* `image`: The source image for segmentation.
|
||||
* `segment_prompt`: A text description of the objects to segment (e.g., "the car", "all people").
|
||||
* `api_key`: Your API key from Google AI Studio.
|
||||
* `model`: The Gemini model to use.
|
||||
* `...other_params`: Controls for temperature, thinking, and seed.
|
||||
* `temperature`: Controls randomness.
|
||||
* `thinking`: Enable thinking process.
|
||||
* `seed`: Seed for reproducibility.
|
||||
* `api_key`: Your API key from Google AI Studio.
|
||||
* `thinking_budget` (Optional): Token budget for thinking.
|
||||
* **Output:**
|
||||
* `mask`: A black and white mask of the segmented objects.
|
||||
|
||||
@@ -108,12 +131,13 @@ Separate audio into different speaker tracks using Gemini.
|
||||
* **Category:** `audio/diarise`
|
||||
* **Inputs:**
|
||||
* `audio`: The input audio to process.
|
||||
* `num_speakers`: The expected number of speakers (default: 2).
|
||||
* `model`: The Gemini model to use (e.g., `gemini-2.5-flash`).
|
||||
* `num_speakers`: The expected number of speakers.
|
||||
* `model`: The Gemini model to use.
|
||||
* `api_key`: Your API key from Google AI Studio.
|
||||
* `thinking`: Enable thinking process (optional).
|
||||
* `thinking_budget`: Token budget for thinking (optional).
|
||||
* `...other_params`: Controls for temperature and seed.
|
||||
* `seed`: Seed for reproducibility.
|
||||
* `temperature`: Controls randomness.
|
||||
* `thinking` (Optional): Enable thinking process.
|
||||
* `thinking_budget` (Optional): Token budget for thinking.
|
||||
* **Output:**
|
||||
* `speaker_1` to `speaker_4`: Audio tracks for up to 4 separated speakers.
|
||||
|
||||
@@ -125,13 +149,44 @@ This node uses OpenAI's API to perform powerful, prompt-based inpainting and edi
|
||||
* **Inputs:**
|
||||
* `image`: The source image to edit.
|
||||
* `mask` (Optional): A black and white mask. The model will edit the **white area** of the mask.
|
||||
* `prompt`: A description of the edit to perform (e.g., "Add a small red boat on the water", "Remove the person on the left").
|
||||
* `prompt`: A description of the edit to perform.
|
||||
* `api_key`: Your API key from OpenAI.
|
||||
* `...other_params`: Various quality and formatting options for the OpenAI API.
|
||||
* **Output:**
|
||||
* `image`: The edited image.
|
||||
|
||||
**Note:** If a mask is provided, the edits will be constrained to the masked region. If no mask is provided, the model will attempt to edit the entire image based on the prompt.
|
||||
### OpenAI LLM
|
||||
|
||||
Access OpenAI's powerful language models for text generation and reasoning.
|
||||
|
||||
* **Category:** `text/generation`
|
||||
* **Inputs:**
|
||||
* `prompt`: The text prompt.
|
||||
* `model`: The OpenAI model to use (e.g., `gpt-4.1`, `o1`, `gpt-5`).
|
||||
* `temperature`: Controls randomness.
|
||||
* `reasoning_effort`: Effort level for reasoning models.
|
||||
* `api_key`: Your OpenAI API key.
|
||||
* `max_output_tokens`: Maximum number of tokens to generate.
|
||||
* `system_instruction` (Optional): System level instructions.
|
||||
* `image` (Optional): Input image for multimodal models.
|
||||
* **Output:**
|
||||
* `response`: The generated text response.
|
||||
|
||||
### OpenAI Text-to-Speech
|
||||
|
||||
Generate high-quality speech from text using OpenAI's TTS models.
|
||||
|
||||
* **Category:** `audio/generation`
|
||||
* **Inputs:**
|
||||
* `text`: The text to convert to speech.
|
||||
* `model`: The TTS model to use (e.g., `gpt-4o-mini-tts`, `tts-1`).
|
||||
* `voice`: The voice to use (e.g., `alloy`, `echo`).
|
||||
* `response_format`: Output audio format.
|
||||
* `speed`: Speaking speed.
|
||||
* `api_key`: Your OpenAI API key.
|
||||
* `instructions` (Optional): Instructions for the model (supported by some models).
|
||||
* **Output:**
|
||||
* `audio`: The generated audio.
|
||||
|
||||
### Google Imagen Generator
|
||||
|
||||
@@ -206,8 +261,6 @@ Generate videos using Google's Veo 2.0 model via the Gemini API. Supports text-t
|
||||
* **Output:**
|
||||
* `frames`: The generated video frames.
|
||||
|
||||
---
|
||||
|
||||
### ElevenLabs TTS
|
||||
|
||||
Generate speech from text using the ElevenLabs API.
|
||||
@@ -249,4 +302,4 @@ Generate speech from text using Google's Gemini TTS models.
|
||||
|
||||
* The [ComfyUI](https://github.com/comfyanonymous/ComfyUI) team for creating such a flexible and powerful platform.
|
||||
* [Google](https://deepmind.google/technologies/gemini/), [OpenAI](https://openai.com/), and [Black Forest Labs](https://www.blackforestlabs.ai/) for developing these incredible models.
|
||||
* [Replicate](https://replicate.com/) for providing easy API access to a wide range of models.
|
||||
* [Replicate](https://replicate.com/) for providing easy API access to a wide range of models.
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "externalapi-helpers"
|
||||
description = "Various ComfyUI nodes for Gemini, Replicate and OpenAI"
|
||||
version = "1.0.4"
|
||||
version = "1.0.5"
|
||||
license = {file = "LICENSE"}
|
||||
# classifiers = [
|
||||
# # For OS-independent nodes (works on all operating systems)
|
||||
|
||||
+2
-1
@@ -4,4 +4,5 @@ numpy
|
||||
torch
|
||||
google-genai
|
||||
opencv-python
|
||||
soundfile
|
||||
soundfile
|
||||
openai
|
||||
Reference in New Issue
Block a user