This custom node seamlessly integrates the Lumina-Next-SFT model into ComfyUI, enabling high-quality image generation using the advanced Lumina text-to-image pipeline. While still under active development, it offers a robust and functional implementation.

Features

Harnesses the power of the Lumina-Next-SFT model for state-of-the-art image generation
Offers a wide range of generation parameters for fine-tuned control
Implements Lumina-specific features including scaling watershed and proportional attention
Incorporates time-aware scaling for improved generation quality
Utilizes an ODE solver for enhanced stability and quality in the generation process
Automatic model downloading for seamless setup
Outputs generated images and working on latent representations (Latent isn't working at the moment)

Installation

Ensure you have ComfyUI installed and properly set up.

Clone this repository into your ComfyUI custom nodes directory:

git clone https://github.com/Excidos/ComfyUI-Lumina-Diffusers.git

The required dependencies will be automatically installed.

NOTE: This installation includes a development branch of diffusers, which may conflict with some existing nodes.

Usage

Launch ComfyUI.
Locate the "Lumina-Next-SFT Diffusers" node in the node selection menu.
Add the node to your workflow.
Connect the necessary inputs and outputs.
Configure the node parameters as desired.
Execute your workflow to generate images.

Parameters

model_path: Path to the Lumina model (default: "Lumina-Next-SFT-diffusers")
prompt: Text prompt for image generation
negative_prompt: Negative text prompt
num_inference_steps: Number of denoising steps (default: 30)
guidance_scale: Classifier-free guidance scale (default: 4.0)
width: Output image width (default: 1024)
height: Output image height (default: 1024)
seed: Random seed for generation (-1 for random)
batch_size: Number of images to generate in one batch (default: 1)
scaling_watershed: Scaling watershed parameter (default: 1.0)
proportional_attn: Enable proportional attention (default: True)
clean_caption: Clean input captions (default: True)
max_sequence_length: Maximum sequence length for text input (default: 256)
t_shift: Time shift parameter for scheduler (default: 4)
solver: ODE solver type (options: "euler", "midpoint", "rk4", default: "midpoint")

Outputs

IMAGE: Generated image(s) in tensor format
LATENT: Latent representation of the generated image(s)

Advanced Features

Time-Aware Scaling

The node now implements time-aware scaling, which adjusts the generation process based on the current timestep. This results in improved image quality, especially for high-resolution outputs.

ODE Solver

An Ordinary Differential Equation (ODE) solver has been integrated into the generation process. This advanced technique enhances the stability and quality of the generated images, particularly for complex prompts or high-resolution outputs.