ComfyUI-EasyLLMPrompt

A ComfyUI custom node that converts a structured scene description into a single optimised Stable Diffusion XL positive prompt using a local LLM.

Zero prompt engineering required. Fill in Subject, Place, Time, Action, and/or Notes; the LLM does the rest.

---

Features

• Simple UI – Five text fields and a few optional settings.
• Two modes – Prompt Mode (SDXL prompt compilation) and Raw Mode (send arbitrary text to the LLM).
• Before/After Shell Hooks – Run arbitrary bash commands before or after the node executes (blocking).
• Shell Input – Pass text content to shell hooks via the $input variable.
• Multiple backends – Ollama, llama.cpp server, OpenAI-compatible (vLLM, LM Studio, LocalAI, etc.).
• Persistent config – Backend, URL, model, temperature, and other settings are saved across ComfyUI sessions.
• Fast – LRU cache deduplicates identical requests; configurable timeout prevents blocking.
• Non‑crashing – All errors are returned as descriptive text; ComfyUI never hangs or crashes.
• No external dependencies – Uses only the Python 3 standard library.

---

Installation

Method	Command
ComfyUI Manager	Search for "Easy LLM Prompt" in the Manager
Comfy Registry	comfy node install easy-llm-prompt
Manual	git clone https://github.com/akumaburn/ComfyUI-EasyLLMPrompt.git

Restart ComfyUI. The node appears in the prompt category (right‑click → Add Node → prompt → Easy LLM Prompt).

No pip install needed. The node uses only Python standard library modules.

---

Usage

1. Add the Easy LLM Prompt node to your workflow.
2. Fill in the scene description fields:
   • Subject – who or what is the focus? (e.g. a young woman with freckles)
   • Place – where does the scene happen? (e.g. cherry blossom garden)
   • Time – time of day, era, or season (e.g. golden hour)
   • Action – what is happening? (e.g. reading a book under a tree)
   • Notes – style guidance or extra context (e.g. Studio Ghibli style)
3. Optionally expand the optional section to configure the LLM backend.
4. Switch between Prompt Mode and Raw Mode using the Mode dropdown in the optional section.
5. Shell Hooks (optional) — see Shell Hooks below.
6. Connect the enhanced_prompt output to your SDXL checkpoint/text encoder.

Example

Field	Input
Subject	a majestic white wolf with heterochromatic eyes
Place	frozen tundra under the northern lights
Time	night, aurora borealis
Action	howling at the sky
Notes	photorealistic, cinematic lighting, detailed fur texture

The LLM might produce:

a majestic white wolf with heterochromatic eyes, detailed fur texture, howling at the sky, frozen tundra under the aurora borealis at night, cinematic lighting, northern lights in the background, photorealistic, award-winning photography, 8K, sharp focus

---

Raw Mode

By default the node runs in Prompt Mode, which composes your scene fields (Subject, Place, Time, Action, Notes) into an SDXL-optimised prompt using the LLM.

Switch to Raw Mode (optional → Mode → raw) to bypass the prompt compiler and send any text you want directly to the LLM:

Field	Raw Mode behaviour
System Prompt	Sent directly to the LLM as the system message
User Message (new)	Sent directly to the LLM as the user message
Subject / Place / Time / Action / Notes	Ignored

Use this when you need the LLM's raw output for tasks other than prompt generation — summarisation, translation, classification, etc.

---

Backend Setup

Ollama (recommended for beginners)

1. Download and install Ollama.
2. Pull a model:

   ollama pull llama3.2
   

3. Leave the Base URL as http://localhost:11434 and set Model to llama3.2.

llama.cpp server

1. Build or download llama.cpp and start the server:

   ./server -m path/to/model.gguf --host 0.0.0.0 --port 8080
   

2. In the node, select llama.cpp, set Base URL to http://localhost:8080, and enter your model name.

LM Studio

1. Load a model in LM Studio and start the local inference server (default port 1234).
2. Select OpenAI Compatible, set Base URL to http://localhost:1234, and enter the model name.

vLLM / LocalAI / other OpenAI-compatible

Start your server, then use the OpenAI Compatible backend with the correct base URL.

---

Shell Hooks

The Before-Run (Shell) and After-Run (Shell) optional fields let you execute arbitrary bash commands before or after the node's main logic runs. Execution is blocking — the node waits for the command to finish before proceeding.

Use cases

• Start/stop a server — launch an LLM backend, wait for it to be ready, then run the node.
• Pre-processing — download a model, prepare input data, or reset state.
• Post-processing — save results, send notifications, or clean up resources.
• Pass dynamic input — use the Shell Input field to pass text content to your shell commands.

Shell Input ($input)

The Shell Input optional field exposes its content to both shell hooks via the $input bash variable, which points to a temporary file containing the text. If the field is empty, $input is not set.

# Example: use the shell input as a prompt for another tool
cat "$input" | some-command

# Or pass it inline
text=$(cat "$input")
echo "$text"

You can connect an upstream node's output to Shell Input to chain data through your workflow, or type directly into the field.

Notes

• The command is written to a temporary file and executed with bash, not /bin/sh.
• Background processes (via &) are supported — the foreground script still blocks.
• Exit codes are not enforced (non-zero exits do not crash the node).
• The temp script and input file are deleted after execution.

Example

pkill -f llama-server 2>/dev/null
(cd /path/to/backend && ./gradlew quarkusDev --console=plain) &
sleep 10

---

Configuration

Settings are persisted in a JSON file at:

• ComfyUI user directory: {ComfyUI}/user/easy_llm_prompt/config.json
• Fallback: ~/.config/easy_llm_prompt/config.json

Setting	Default	Description
default_backend	ollama	Backend to use by default
default_base_url	http://localhost:11434	Default server URL
default_model	llama3.2	Default model name
default_temperature	0.7	Default temperature
default_max_tokens	512	Default max output length
timeout	60	HTTP request timeout (seconds)
cache_size	100	Max LRU cache entries

You can edit this file manually or change settings per-node through the UI (changes are not saved to the config file by the UI – only per‑workflow).

---

Outputs

Output	Type	Description
enhanced_prompt	STRING	The prompt generated by the LLM
debug_info	STRING	Backend, model, timing, and cache status

Connect enhanced_prompt to your text encoder and debug_info to a ShowText node for diagnostics.

---

Error Messages

Message	Likely cause
ERROR: Connection refused	The LLM server is not running.
ERROR: Could not resolve hostname	The URL is incorrect.
ERROR: Request timed out	The server is too slow; increase the timeout in config.
ERROR: Authentication failed	The endpoint requires an API key.
ERROR: Endpoint not found	The URL path is wrong (e.g. missing /v1).
ERROR: Unexpected API response	The model is not a chat model or returned garbage.

---

Troubleshooting

The node returns an empty or nonsensical prompt.
Try a different model. Small models (< 3B parameters) often struggle with instruction following.

The node is slow.
Reduce max_tokens to 256 or lower. Use a smaller/faster model. Ensure your LLM server runs on the same machine or a fast local network.

The output contains explanation text.
Make sure the System Prompt field is empty (the default instructs the LLM to output only the prompt). If you use a custom system prompt, include a "Do not explain anything" directive.

Settings don't persist.
The node reads the config file on startup but does not write back. To change defaults, edit the JSON file directly.

---

Project Structure

ComfyUI-EasyLLMPrompt/
├── __init__.py              # ComfyUI node registration
├── pyproject.toml           # Package metadata
├── requirements.txt         # No external dependencies
├── .gitignore
└── easy_llm_prompt/
    ├── __init__.py          # Package version
    ├── node.py              # EasyLLMPromptNode class (Prompt Mode, Raw Mode, Shell Hooks)
    ├── llm_backends.py      # LLM provider implementations
    ├── prompt_builder.py    # System prompt & user message assembly
    ├── cache.py             # LRU request cache
    ├── config.py            # Persistent settings
    └── utils.py             # HTTP helpers, URL normalisation, error handling

---

License

Apache 2.0