ComfyUI Qwen GGUF
Run Qwen GGUF models inside ComfyUI through official llama.cpp release binaries.
This node is intentionally small: it discovers local .gguf models from
ComfyUI/models/LLM, downloads the llama.cpp CLI binary on first use, and calls
llama-mtmd-cli as a subprocess.
Features
- Qwen text generation with GGUF models
- Optional image input through llama.cpp multimodal support
- Separate
RESPONSEandTHINKINGoutputs - System prompt presets from text files
- Recursive model discovery from
ComfyUI/models/LLM - Automatic llama.cpp binary download for Windows x64 CUDA 13
- Advanced escape hatch for extra llama.cpp CLI flags
Installation
Clone or copy this folder into your ComfyUI custom nodes directory:
ComfyUI/custom_nodes/ComfyUI-Qwen-gguf
Restart ComfyUI. The node appears under:
Qwen/GGUF -> Qwen GGUF (llama.cpp)
No Python package install is required for the basic node. ComfyUI already ships with the runtime pieces used here. Pillow and NumPy are used when an image input is connected.
llama.cpp Auto Download
On the first node execution, the extension downloads official llama.cpp release assets from:
https://github.com/ggml-org/llama.cpp/releases
For v1, automatic download supports:
Windows x64 + CUDA 13
The node downloads both release archives:
llama-*-bin-win-cuda-13*-x64.zip
cudart-llama-bin-win-cuda-13*-x64.zip
Recent llama.cpp releases may name the main Windows CUDA 13 binary with the
exact CUDA runtime version, for example llama-b8838-bin-win-cuda-13.1-x64.zip.
They are extracted into:
ComfyUI/custom_nodes/ComfyUI-Qwen-gguf/vendor/llama.cpp/<release-tag>/win-x64-cuda13
Later runs reuse the same folder when the required files are present
(llama-mtmd-cli.exe, ggml-cuda.dll, and cudart64_13.dll). If any required
file is missing, the node treats the install as incomplete and downloads the
archives again.
Linux, macOS, CPU-only, and other CUDA builds are not auto-downloaded yet. The platform layer is kept isolated so those targets can be added cleanly later.
Model Placement
Put your Qwen GGUF files anywhere under:
ComfyUI/models/LLM
Example:
ComfyUI/models/LLM/My-Qwen-Model/model-q4_k_m.gguf
ComfyUI/models/LLM/My-Qwen-Model/mmproj-BF16.gguf
The model dropdown shows recursive .gguf files, excluding files with
mmproj in the filename. The mmproj dropdown shows none plus recursive
.gguf files with mmproj in the filename.
The node downloads llama.cpp binaries only. It does not download Qwen model weights.
System Prompt Presets
Create text files in:
ComfyUI/models/LLM/prompts
Example:
ComfyUI/models/LLM/prompts/captioner.txt
Each top-level .txt file appears in the system_prompt dropdown. Choose
None to run without a system prompt.
Node Inputs
| Input | Description |
|---|---|
model |
GGUF model file from ComfyUI/models/LLM. |
mmproj |
Vision projector GGUF. Required when using image input. |
system_prompt |
Prompt preset from models/LLM/prompts, or None. |
prompt |
User prompt sent to the model. |
max_tokens |
Maximum generated tokens. |
temperature |
Sampling temperature. Lower values are more deterministic. |
top_p |
Nucleus sampling threshold. |
top_k |
Top-K sampling cutoff. |
repeat_penalty |
Penalty for repeated tokens. |
ctx_size |
llama.cpp context size. |
memory_mode |
Advanced memory placement mode. auto passes no layer flags. |
n_gpu_layers |
Used in GPU layer modes. Passes --gpu-layers / -ngl. |
n_cpu_moe_layers |
Used in CPU MoE modes. Passes --n-cpu-moe. |
seed |
Random seed. Use -1 for a random seed. |
timeout_seconds |
Maximum runtime before the subprocess is stopped. |
image |
Optional ComfyUI image input. Uses the first image in a batch. |
extra_args |
Advanced llama.cpp CLI flags appended to the command. |
Every input includes an in-node tooltip.
Node Outputs
| Output | Description |
|---|---|
RESPONSE |
Final model response with Qwen thinking blocks removed. |
THINKING |
Extracted <think>...</think> reasoning when present in model output. |
PERF |
llama.cpp llama_perf_context_print lines. |
Both outputs include ComfyUI output tooltips.
Image Input and mmproj
For image workflows, connect an IMAGE input and choose the matching mmproj
file. The node saves the first image in the batch as a temporary PNG and passes
it to llama.cpp with --image.
If no image is connected, the node runs through llama-completion as text-only
and does not pass --mmproj, even when a projector is selected in the dropdown.
Image workflows run through llama-mtmd-cli, where --mmproj is required.
Troubleshooting
No models appear
Place at least one .gguf model under:
ComfyUI/models/LLM
Then refresh or restart ComfyUI.
Image input fails
Make sure mmproj is not set to none and that the projector belongs to the
same model family as the selected GGUF model.
Download fails
Check your internet connection and GitHub access. You can delete the incomplete
folder under vendor/llama.cpp and run the node again.
Unsupported platform
Automatic binary download currently supports Windows x64 CUDA 13 only. Other platforms need a future platform mapping entry.
llama.cpp errors
Use extra_args only for flags supported by your downloaded llama.cpp build.
If generation times out, increase timeout_seconds or lower max_tokens.