Initial commit of ComfyUI-FLUX-TOGETHER-API project

2024-10-23 16:14:04 +01:00
commit a7ee8a5567
9 changed files with 528 additions and 0 deletions
@@ -0,0 +1,38 @@
+# Ignore Mac system files
+.DS_Store
+
+# Ignore Windows system files
+Thumbs.db
+ehthumbs.db
+
+# Ignore Node.js dependencies
+node_modules/
+
+# Ignore Python cache files
+__pycache__/
+*.py[cod]
+
+# Ignore Java build directories
+/bin/
+/out/
+/target/
+
+# Ignore log files
+*.log
+
+# Ignore environment variables file
+.env
+
+# Ignore compiled files
+*.o
+*.class
+
+# Ignore IDE specific files
+.idea/
+.vscode/
+*.sublime-project
+*.sublime-workspace
+
+# Ignore temporary files
+*.tmp
+*.temp
@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 BZcreativ
+Portions of this code are inspired by ComfyUI-FLUX-BFL-API (https://github.com/gelasdev/ComfyUI-FLUX-BFL-API)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
@@ -0,0 +1,81 @@
+# ComfyUI Together.ai FLUX API Node
+
+A custom node implementation for ComfyUI that integrates with Together.ai's FLUX image generation models. This project is inspired by and adapted from [ComfyUI-FLUX-BFL-API](https://github.com/gelasdev/ComfyUI-FLUX-BFL-API) to work with the Together.ai API.
+
+## Features
+
+- Direct integration with Together.ai's FLUX models
+- Support for FLUX.1-schnell-Free model
+- Configurable parameters including steps, guidance scale, and dimensions
+- Negative prompt support
+- Error handling and retry mechanisms
+
+## Installation
+
+1. Clone this repository into your ComfyUI custom_nodes directory:
+```bash
+cd ComfyUI/custom_nodes
+git clone https://github.com/BZcreativ/ComfyUI-FLUX-TOGETHER-API.git
+```
+
+2. Install the required dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+3. Create a `config.ini` file in the root directory with your Together.ai API key:
+```ini
+[API]
+together_api_key = your_api_key_here
+```
+
+## Configuration
+
+1. Get your API key from [Together.ai](https://together.ai)
+2. Copy the `config.ini.example` to `config.ini`
+3. Add your API key to the configuration file
+
+## Usage
+
+1. Start ComfyUI
+2. Find the "Together API Node" in the node browser
+3. Configure the parameters:
+   - Prompt: Your image generation prompt
+   - Negative Prompt: Elements to avoid in the generation
+   - Steps: Generation steps (1-100)
+   - Width: Image width (512-2048)
+   - Height: Image height (512-2048)
+   - Seed: Generation seed
+   - CFG: Guidance scale (0.0-20.0)
+
+For detailed usage instructions, see [USAGE.md](USAGE.md)
+
+## Parameters
+
+| Parameter | Type | Range | Default | Description |
+|-----------|------|--------|---------|-------------|
+| prompt | string | - | "" | Main generation prompt |
+| negative_prompt | string | - | "" | Elements to avoid |
+| steps | integer | 1-100 | 20 | Number of generation steps |
+| width | integer | 512-2048 | 1024 | Image width |
+| height | integer | 512-2048 | 1024 | Image height |
+| seed | integer | 0-MAX_INT | 0 | Generation seed |
+| cfg | float | 0.0-20.0 | 7.0 | Guidance scale |
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## Credits
+
+- This project is inspired by and adapted from [ComfyUI-FLUX-BFL-API](https://github.com/gelasdev/ComfyUI-FLUX-BFL-API)
+- Together.ai for providing the FLUX API
+- ComfyUI team for the amazing framework
+
+## Author
+
+Created by [BZcreativ](https://github.com/BZcreativ)
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
@@ -0,0 +1,129 @@
+# ComfyUI Together.ai FLUX API Node Usage Guide
+
+## Setup
+
+1. Ensure you have a Together.ai account and API key
+2. Configure your API key in `config.ini`
+3. Install all required dependencies
+
+## Node Configuration
+
+### Input Parameters
+
+#### Required Parameters:
+- **Prompt** (String)
+  - Your main generation prompt
+  - Supports multiline input
+  - Be specific and detailed for best results
+
+- **Negative Prompt** (String)
+  - Elements you want to avoid in the generation
+  - Supports multiline input
+  - Leave empty if not needed
+
+- **Steps** (Integer)
+  - Range: 1-100
+  - Default: 20
+  - Higher values generally produce better quality but take longer
+  - Recommended range: 20-50 for most use cases
+
+- **Width** (Integer)
+  - Range: 512-2048
+  - Default: 1024
+  - Must be a multiple of 8
+  - Common values: 512, 768, 1024
+
+- **Height** (Integer)
+  - Range: 512-2048
+  - Default: 1024
+  - Must be a multiple of 8
+  - Common values: 512, 768, 1024
+
+- **Seed** (Integer)
+  - Range: 0 to max 64-bit integer
+  - Default: 0
+  - Use specific seeds to reproduce results
+  - 0 or -1 for random seed
+
+- **CFG (Guidance Scale)** (Float)
+  - Range: 0.0-20.0
+  - Default: 7.0
+  - Controls how closely the image follows the prompt
+  - Recommended range: 5.0-10.0
+
+### Output
+
+The node outputs a single image tensor compatible with other ComfyUI nodes.
+
+## Best Practices
+
+1. **Prompt Engineering**
+   - Be specific and detailed in your prompts
+   - Use descriptive adjectives
+   - Include style references when needed
+
+2. **Performance**
+   - Start with lower step counts (20-30) for testing
+   - Increase steps for final generations
+   - Use reasonable image dimensions (1024x1024 is standard)
+
+3. **Error Handling**
+   - Check console for error messages
+   - Verify API key is correctly configured
+   - Ensure parameters are within valid ranges
+
+## Common Workflows
+
+### Basic Image Generation
+1. Add Together API Node to workspace
+2. Connect to a Load Image node
+3. Configure prompt and basic parameters
+4. Execute workflow
+
+### Advanced Usage
+1. Combine with other ComfyUI nodes
+2. Use seed control for consistent results
+3. Experiment with guidance scale for style control
+
+## Troubleshooting
+
+### Common Issues
+
+1. **API Key Errors**
+   - Verify key in config.ini
+   - Check API key validity
+   - Ensure proper formatting
+
+2. **Generation Errors**
+   - Verify parameter ranges
+   - Check prompt length
+   - Monitor API rate limits
+
+3. **Image Quality Issues**
+   - Adjust step count
+   - Modify guidance scale
+   - Refine prompt
+
+## Examples
+
+### Basic Prompt Example
+```
+A beautiful landscape with mountains and lakes, cinematic lighting, high detail
+```
+
+### Advanced Prompt Example
+```
+A stunning mountain landscape at sunset, volumetric lighting, 
+golden hour, ultra detailed, professional photography, 
+8k resolution, artistic composition
+```
+
+### Negative Prompt Example
+```
+blur, haze, low quality, distortion, bad composition, 
+oversaturated, unrealistic lighting
+```
+
+## Support
+
+For issues and feature requests, please use the GitHub issue tracker.
@@ -0,0 +1,18 @@
+import importlib.util
+import importlib
+
+node_list = [
+"api_node",
+]
+
+NODE_CLASS_MAPPINGS = {}
+NODE_DISPLAY_NAME_MAPPINGS = {}
+
+for module_name in node_list:
+    imported_module = importlib.import_module(f".nodes.{module_name}", __name__)
+
+    NODE_CLASS_MAPPINGS = {**NODE_CLASS_MAPPINGS, **imported_module.NODE_CLASS_MAPPINGS}
+    NODE_DISPLAY_NAME_MAPPINGS = {**NODE_DISPLAY_NAME_MAPPINGS, **imported_module.NODE_DISPLAY_NAME_MAPPINGS}
+
+
+__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
@@ -0,0 +1,3 @@
+[API]
+API_KEY = XXXXXX
+BASE_URL = https://api.together.xyz/v1
@@ -0,0 +1,2 @@
+[API]
+together_api_key = your_api_key_here
@@ -0,0 +1,231 @@
+import io
+import os
+import configparser
+from enum import Enum
+from urllib.parse import urljoin
+from PIL import Image
+import numpy as np
+import torch
+from together import Together
+import base64
+import json
+from tenacity import retry, stop_after_attempt, wait_exponential
+
+class Status(Enum):
+    TASK_NOT_FOUND = "Task not found"
+    PENDING = "pending"
+    REQUEST_MODERATED = "Request Moderated"
+    CONTENT_MODERATED = "Content Moderated"
+    READY = "completed"
+    ERROR = "error"
+
+class ConfigLoader:
+    def __init__(self):
+        current_dir = os.path.dirname(os.path.abspath(__file__))
+        parent_dir = os.path.dirname(current_dir)
+        config_path = os.path.join(parent_dir, "config.ini")
+        self.config = configparser.ConfigParser()
+        self.config.read(config_path)
+        self.set_api_key()
+
+    def get_key(self, section, key):
+        try:
+            return self.config[section][key]
+        except KeyError:
+            raise KeyError(f"{key} not found in section {section} of config file.")
+
+    def set_api_key(self):
+        try:
+            api_key = self.get_key('API', 'API_KEY')
+            os.environ["TOGETHER_API_KEY"] = api_key
+        except KeyError as e:
+            print(f"Error: {str(e)}")
+
+config_loader = ConfigLoader()
+
+class BaseFlux:
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "generate_image"
+    CATEGORY = "Together.ai"
+
+    def __init__(self):
+        self.client = Together()
+
+    def process_result(self, result):
+        try:
+            print(f"Debug - Result type: {type(result)}")
+            print(f"Debug - Result content: {result}")
+            
+            if isinstance(result, dict) and 'data' in result:
+                img_data = result['data'][0]['b64_json']
+            elif hasattr(result, 'data') and hasattr(result.data[0], 'b64_json'):
+                img_data = result.data[0].b64_json
+            else:
+                raise ValueError("Unexpected response format")
+
+            img_bytes = base64.b64decode(img_data)
+            img = Image.open(io.BytesIO(img_bytes))
+            img_array = np.array(img).astype(np.float32) / 255.0
+            img_tensor = torch.from_numpy(img_array)[None,]
+            return (img_tensor,)
+        except Exception as e:
+            print(f"Error processing image result: {str(e)}")
+            return self.create_blank_image()
+
+    def create_blank_image(self):
+        blank_img = Image.new('RGB', (512, 512), color='black')
+        img_array = np.array(blank_img).astype(np.float32) / 255.0
+        img_tensor = torch.from_numpy(img_array)[None,]
+        return (img_tensor,)
+
+    def check_multiple_of_32(self, width, height):
+        if width % 32 != 0 or height % 32 != 0:
+            raise ValueError(f"Width {width} and height {height} must be multiples of 32.")
+
+    def generate_image(self, model_path, arguments):
+        self.check_multiple_of_32(arguments["width"], arguments["height"])
+
+        try:
+            response = self.client.images.generate(
+                model=model_path,
+                prompt=arguments["prompt"],
+                width=arguments["width"],
+                height=arguments["height"],
+                steps=arguments["steps"],
+                n=1,
+                response_format="b64_json",
+                **{k: v for k, v in arguments.items() if k not in ["prompt", "width", "height", "steps"]}
+            )
+            return self.process_result(response)
+        except Exception as e:
+            print(f"Error generating image: {str(e)}")
+            return self.create_blank_image()
+
+class FluxPro11(BaseFlux):
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "prompt": ("STRING", {"default": "", "multiline": True}),
+                "width": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "height": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "prompt_upsampling": ("BOOLEAN", {"default": True}),
+                "steps": ("INT", {"default": 1, "min": 1, "max": 1}),
+                "safety_tolerance": (["1", "2", "3", "4", "5", "6"], {"default": "2"}),
+            },
+            "optional": {
+                "seed": ("INT", {"default": -1})
+            }
+        }
+
+    def generate_image(self, prompt, width, height, prompt_upsampling, steps, safety_tolerance, seed=-1):
+        arguments = {
+            "prompt": prompt,
+            "width": width,
+            "height": height,
+            "prompt_upsampling": prompt_upsampling,
+            "steps": steps,
+            "safety_tolerance": safety_tolerance
+        }
+        if seed != -1:
+            arguments["seed"] = seed
+        return super().generate_image("black-forest-labs/FLUX.1.1-pro", arguments)
+
+class FluxDev(BaseFlux):
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "prompt": ("STRING", {"default": "", "multiline": True}),
+                "width": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "height": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "steps": ("INT", {"default": 4, "min": 1, "max": 4}),
+                "prompt_upsampling": ("BOOLEAN", {"default": True}),
+                "safety_tolerance": (["1", "2", "3", "4", "5", "6"], {"default": "2"}),
+                "guidance": ("FLOAT", {"default": 3.0, "min": 0.1, "max": 10.0}),
+            },
+            "optional": {
+                "seed": ("INT", {"default": -1})
+            }
+        }
+
+    @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
+    def generate_image(self, prompt, width, height, steps, prompt_upsampling, safety_tolerance, guidance, seed=-1):
+        arguments = {
+            "prompt": prompt,
+            "width": width,
+            "height": height,
+            "steps": steps,
+            "prompt_upsampling": prompt_upsampling,
+            "safety_tolerance": safety_tolerance,
+            "guidance_scale": guidance  # Changed from guidance to guidance_scale to match API expectations
+        }
+        if seed != -1:
+            arguments["seed"] = seed
+        
+        try:
+            # Override the base class method to handle the response directly
+            self.check_multiple_of_32(width, height)
+            response = self.client.images.generate(
+                model="black-forest-labs/FLUX.1-schnell-Free",
+                prompt=prompt,
+                width=width,
+                height=height,
+                steps=steps,
+                n=1,
+                response_format="b64_json",
+                guidance_scale=guidance,  # Explicitly pass guidance_scale
+                seed=seed if seed != -1 else None,
+                safety_tolerance=safety_tolerance,
+                prompt_upsampling=prompt_upsampling
+            )
+            return self.process_result(response)
+        except Exception as e:
+            print(f"Error generating image: {str(e)}")
+            return self.create_blank_image()
+
+class FluxPro(BaseFlux):
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "prompt": ("STRING", {"default": "", "multiline": True}),
+                "width": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "height": ("INT", {"default": 1024, "min": 256, "max": 1440}),
+                "steps": ("INT", {"default": 4, "min": 1, "max": 40}),
+                "prompt_upsampling": ("BOOLEAN", {"default": True}),
+                "safety_tolerance": (["1", "2", "3", "4", "5", "6"], {"default": "2"}),
+                "guidance": ("FLOAT", {"default": 2.5, "min": 0.1, "max": 10.0}),
+                "interval": ("INT", {"default": 2, "min": 1, "max": 10}),
+            },
+            "optional": {
+                "seed": ("INT", {"default": -1})
+            }
+        }
+
+    def generate_image(self, prompt, width, height, steps, prompt_upsampling, safety_tolerance, guidance, interval, seed=-1):
+        arguments = {
+            "prompt": prompt,
+            "width": width,
+            "height": height,
+            "steps": steps,
+            "prompt_upsampling": prompt_upsampling,
+            "safety_tolerance": safety_tolerance,
+            "guidance": guidance,
+            "interval": interval
+        }
+        if seed != -1:
+            arguments["seed"] = seed
+        return super().generate_image("black-forest-labs/FLUX.1-pro", arguments)
+
+NODE_CLASS_MAPPINGS = {
+    "FluxPro11_TOGETHER": FluxPro11,
+    "FluxDev_TOGETHER": FluxDev,
+    "FluxPro_TOGETHER": FluxPro
+}
+
+NODE_DISPLAY_NAME_MAPPINGS = {
+    "FluxPro11_TOGETHER": "Flux Pro 1.1 (TOGETHER)",
+    "FluxDev_TOGETHER": "Flux Dev (TOGETHER)",
+    "FluxPro_TOGETHER": "Flux Pro (TOGETHER)"
+}
@@ -0,0 +1,4 @@
+requests>=2.31.0
+Pillow>=10.0.0
+numpy>=1.24.0
+configparser>=5.3.0