image-caption-comfyui

• image-caption-comfyui
  • Setup
  • Example Workflow
  • Pretrained Image Caption Models
    • Models
  • Variables
    • Image Caption Node
    • Insert Prompt Node
  • Troubleshooting
  • Contributing
  • Example Output

Image caption node for ComfyUI. You can load your image caption model and generate prompts with the given picture.

Setup

• Clone the repository to the custom_nodes folder

• Run ComfyUI

• Place the folder which contains your model under the models/image_captioners folder

• Click Refresh button in ComfyUI, if it didn't work restart ComfyUI

• Processor has to be in the folder

Example Workflow

Pretrained Image Caption Models

• You can find the models in this link

• For to use the pretrained model follow these steps:

  • Download the model and unzip to models/image_captioners folder.
  • Click Refresh button in ComfyUI
  • Then select the image caption model with the node's model_name variable (If you can't see the generator, restart ComfyUI).

Models

• female_image_caption_blip | (Training In Process)
  • Base model
  • using Salesforce/blip-image-captioning-base

Variables

Image Caption Node

Variable Names	Definitions
model_name	Folder name that contains the model
min_new_tokens	The minimum numbers of tokens to generate, ignoring the number of tokens in the prompt.
max_new_tokens	The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.
num_beams	Number of steps for each search path
repetition_penalty	The parameter for repetition penalty. 1.0 means no penalty

• For more information, follow this link.
• Check this link for text generation strategies.

Insert Prompt Node

Variable Names	Definitions
prompt_string	Want to be inserted prompt. It is replaced with {prompt_string} part in the prompt_format variable
prompt_format	New prompts with including prompt_string variable's value with {prompt_string} syntax. For example, 1girl, solo, {prompt_string}. The {prompt_string} syntax will be added anywhere in the string.

• After including the node, change the prompt_string variable to input

Troubleshooting

• If the below solutions are not fixed your issue please create an issue with bug label

Contributing

• Contributions are welcome. If you have an idea and want to implement it by yourself please follow these steps:

  1. Create a fork
  2. Pull request the fork with the description that explaining the new feature

• If you have an idea but don't know how to implement it, please create an issue with enhancement label.

• The contributing can be done in several ways. You can contribute to code or to README file.

Example Output