ComfyUI Plugin: Advanced Prompt Generation and Image Analysis

comfyui_dagthomas

This plugin extends ComfyUI with advanced prompt generation capabilities and image analysis using GPT-4 Vision. It includes the following components:

Classes

1. PromptGenerator

A versatile prompt generator for text-to-image AI systems.

Features:

• Generates prompts based on various customizable parameters
• Supports different art forms, photography styles, and digital art
• Allows for random selection or specific choices for each parameter
• Outputs separate prompts for different model components (e.g., CLIP, T5)

"subject" input field can be changed to support your style or lora, just add your subject and it will overwrite "man, woman ..." etc.

"custom" input field will add a prompt to the start of the prompt string. For loading styles

GPT4VisionNode

Analyzes images using OpenAI's GPT-4 Vision model.

Features:

• Accepts image input and generates detailed descriptions
• Supports custom base prompts
• Offers options for "happy talk" (detailed descriptions) or simple outputs
• Includes compression options to limit output length
• Ability to create posters

Workflow

There is a toggle to create movie posters (08/04/24)

GPT4MiniNode

Generates text using OpenAI's GPT-4 model based on input text.

Features:

• Accepts text input and generates enhanced descriptions
• Supports custom base prompts
• Offers options for "happy talk" (detailed descriptions) or simple outputs
• Includes compression options to limit output length

Workflow

OllamaNode

Generates text using custom Ollama based on input text.

Features:

• Accepts text input and generates enhanced descriptions
• Supports custom base prompts
• Offers options for "happy talk" (detailed descriptions) or simple outputs
• Includes compression options to limit output length

Workflow

Pure Florence workflow:

• You can also use a pure local Florence workflow without any of the others. The prompt will have some bloat, but works fine with Flux

Workflow

PGSD3LatentGenerator

Generates latent representations for use in Stable Diffusion 3 pipelines.

Features:

• Creates latent tensors with specified dimensions
• Supports batch processing
• Automatically adjusts dimensions to maintain a consistent megapixel count

Usage

These classes can be integrated into ComfyUI workflows to enhance prompt generation, image analysis, and latent space manipulation for advanced AI image generation pipelines.

Requirements

• OpenAI API key (for GPT4VisionNode and GPT4MiniNode)
• ComfyUI environment
• Additional dependencies as specified in the import statements

Notes

• Ensure that your OpenAI API key is set in the environment variables
• Some classes may require additional data files (JSON) for their functionality
• Refer to the individual class documentation for specific usage instructions and input types

Following were generated with Flux Dev - 03/08/2024

Following were generated with SD3 Medium - 06/15/2024