ComfyUI-QwenVL-Utils

A comprehensive and modular QwenVL integration for ComfyUI, providing advanced vision-language capabilities with support for both HuggingFace Transformers and GGUF models. This extension consolidates features from multiple QwenVL implementations while introducing enhanced error handling, attention backend optimization, and a clean, maintainable codebase.

Credits & Acknowledgments

This project builds upon and consolidates features from multiple excellent QwenVL implementations:

Original Implementations

• ComfyUI-QwenVL by 1038lab

  • GGUF backend integration
  • System prompt templates
  • Advanced parameter controls
  • Comprehensive model support

• ComfyUI_Qwen2-VL-Instruct by IuvenisSapiens

  • Initial ComfyUI integration
  • Multi-image and video support
  • Path node implementation
  • Clean workflow design

Supported Models

Qwen3.5+ Unified Vision-Language Models

Qwen3.5, Qwen3.6, and Qwen3.8 use unified thinking/instruct mode — a single model supports both reasoning and direct instruction-following, controlled by the enable_thinking toggle in the node UI. No separate -Instruct and -Thinking model files are needed.

Model	Size	Architecture	VRAM (FP16)	VRAM (8-bit)	VRAM (4-bit)
Qwen3.5-9B	9B	Dense (Hybrid Gated Delta Net)	~20GB	~12GB	~7GB
Qwen3.5-27B	27B	Dense (Hybrid Gated Delta Net)	~56GB	~30GB	~18GB
Qwen3.5-35B-A3B	35B total / 3B active	MoE (Hybrid Gated Delta Net)	~72GB	~36GB	~20GB
Qwen3.6-27B	27B	Dense unified multimodal	~56GB	~28GB	~16GB
Qwen3.6-35B-A3B	35B total / 3B active	MoE unified multimodal	~72GB	~36GB	~20GB
Qwen3.8-27B	27B	Dense unified multimodal	~56GB	~28GB	~16GB

FP8 variants are available for Qwen3.5-9B/27B/35B-A3B, Qwen3.6-27B/35B-A3B, and Qwen3.8-27B. The 27B FP8 entries are estimated at ~28GB and the 35B-A3B entries at ~36GB.

Note

: Qwen3.5 uses a novel hybrid architecture combining Gated Delta Networks with sparse MoE. The 35B-A3B variant activates only ~3B parameters per token, making it very memory-efficient despite 35B total parameters.

Qwen3-VL / Qwen2.5-VL Models

Model	Size	Features	VRAM (FP16)	VRAM (8-bit)	VRAM (4-bit)
Qwen3-VL-2B-Instruct	2B	General VL	~4GB	~2.5GB	~1.5GB
Qwen3-VL-2B-Thinking	2B	CoT reasoning	~4GB	~2.5GB	~1.5GB
Qwen3-VL-4B-Instruct	4B	Balanced	~6GB	~3.5GB	~2GB
Qwen3-VL-4B-Thinking	4B	CoT reasoning	~6GB	~3.5GB	~2GB
Qwen3-VL-8B-Instruct	8B	High quality	~12GB	~7GB	~4.5GB
Qwen3-VL-8B-Thinking	8B	Advanced CoT	~12GB	~7GB	~4.5GB
Qwen3-VL-32B-Instruct	32B	Best quality	~28GB	~14GB	~8.5GB
Qwen3-VL-32B-Thinking	32B	Complex reasoning	~28GB	~14GB	~8.5GB
Qwen2.5-VL-3B-Instruct	3B	Previous gen	~6GB	~3.5GB	~2GB
Qwen2.5-VL-7B-Instruct	7B	Previous gen	~15GB	~8.5GB	~5GB

FP8 Pre-Quantized Models (40-series GPU recommended):

• Qwen3-VL-2B-*-FP8: ~2.5GB VRAM
• Qwen3-VL-4B-*-FP8: ~2.5GB VRAM
• Qwen3-VL-8B-*-FP8: ~7.5GB VRAM
• Qwen3-VL-32B-*-FP8: ~24GB VRAM

GGUF Quantized Models

The unified Qwen3.5+ GGUF models are sourced from unsloth; older Qwen-VL entries retain their configured upstream repositories.

Model	Source	Variants	Features
Qwen3.5 (Unified Thinking/Instruct)
Qwen3.5-9B-GGUF	unsloth	Q4_K_M, Q8_0, BF16	Unified thinking + instruct
Qwen3.5-27B-GGUF	unsloth	Q4_K_M, Q8_0, BF16	Unified thinking + instruct
Qwen3.5-35B-A3B-GGUF	unsloth	Q4_K_M, Q8_0, BF16	MoE, unified thinking + instruct
Qwen3.6 / Qwen3.8 (Unified Thinking/Instruct)
Qwen3.6-27B-GGUF	unsloth	IQ4/Q4/Q5/Q6/Q8, UD quants, split BF16	Unified thinking + instruct
Qwen3.6-35B-A3B-GGUF	unsloth	MXFP4_MOE/Q8, UD quants, split BF16	MoE, unified thinking + instruct
Qwen3.8-27B-GGUF	unsloth	Q4/Q8, UD quants, split BF16	Generic MTMD, unified thinking + instruct
Qwen3.6-27B-GGUF	lmstudio-community	Q4_K_M, Q6_K	Unified thinking + instruct
Qwen3.6-35B-A3B-GGUF	lmstudio-community	Q4_K_M	MoE, unified thinking + instruct
Qwen3.8-27B-GGUF	lmstudio-community	Q4_K_M, Q6_K, Q8_0	Generic MTMD, unified thinking + instruct
Qwen3-VL
Qwen3-VL-2B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Instruct tuned
Qwen3-VL-4B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Instruct tuned
Qwen3-VL-8B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Instruct tuned
Qwen3-VL-4B-Thinking-GGUF	unsloth	Q4_K_M, Q8_0	Thinking mode
Qwen3-VL-8B-Thinking-GGUF	unsloth	Q4_K_M, Q8_0	Thinking mode
Qwen2.5-VL
Qwen2.5-VL-3B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Previous gen
Qwen2.5-VL-7B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Previous gen
Qwen2.5-VL-32B-Instruct-GGUF	unsloth	Q4_K_M, Q8_0	Previous gen

GGUF Quantization Guide:

• Q4_K_M: ~3-4GB VRAM, good balance of quality and speed
• Q8_0: ~5-7GB VRAM, better quality
• BF16: near full precision and much larger than quantized variants. Qwen3.6/Qwen3.8 BF16 downloads are split into two colocated shards under BF16/; the node downloads both automatically.

Note

: Qwen3.5+ GGUF support requires the JamePeng llama-cpp-python fork >= 0.3.49. Qwen3.5/Qwen3.6 use Qwen35ChatHandler; Qwen3.8 uses GenericMTMDChatHandler.

Locally mirrored LM Studio Community models use source-qualified dropdown names and are stored under models/LLM/GGUF/lmstudio-community/<repo-name>/, with each repository's matching mmproj beside its model files.

Installation

Method 1: ComfyUI Manager (Recommended)

1. Open ComfyUI Manager
2. Search for "QwenVL Utils"
3. Click "Install"
4. Restart ComfyUI

Method 2: Manual Installation

1. Clone this repository into your ComfyUI/custom_nodes directory:

   cd ComfyUI/custom_nodes
   git clone https://github.com/AkihaTatsu/ComfyUI-QwenVL-Utils.git
   

2. Install core dependencies with ComfyUI's Python interpreter:

   cd ComfyUI-QwenVL-Utils
   python -m pip install -r requirements.txt
   python install.py
   

3. (Optional) Install optional features:

   # For 4-bit/8-bit quantization
   pip install bitsandbytes>=0.41.0
   
   # For Flash Attention 2 (Ampere+ GPU)
   pip install flash-attn --no-build-isolation
   
   # For SageAttention (experimental)
   pip install sageattention
   
   # Install all optional dependencies
   pip install -e ".[all]"
   

4. Restart ComfyUI

GGUF Setup (Optional)

For GGUF model support with vision capabilities:

• ComfyUI Manager invokes install.py automatically. Existing compatible builds are preserved.
• The installer selects CUDA on Windows/Linux, Metal on Apple Silicon, Vulkan when its SDK is detected, and CPU otherwise.
• CUDA and Metal use matching pre-built wheels when available; Vulkan, CPU, and unavailable wheel combinations are built automatically from the pinned JamePeng llama-cpp-python fork.
• Source builds require a C/C++ compiler and the selected backend's toolkit. The installer does not install system toolchains.
• To override detection, set QWENVL_LLAMA_CPP_BACKEND to auto, cuda, metal, vulkan, cpu, or skip, then reinstall/update the node or run install.py with the ComfyUI Python.
• A compatible build must be version 0.3.49 or newer and provide Qwen3VLChatHandler, Qwen35ChatHandler, and GenericMTMDChatHandler.
• MTP sidecars are intentionally not loaded; these entries use standard autoregressive inference.

Node Overview

QwenVL (Basic)

Simplified interface for quick vision-language tasks. Qwen3.5+ models automatically use their recommended general-task sampling parameters; older model families retain the legacy defaults.

Parameters:

• model_name: Model selection (HF or [GGUF] prefixed)
• enable_thinking: Toggle Thinking/Instruct mode for unified Qwen3.5/3.6/3.8 models (see Thinking Mode)
• reasoning_effort: Qwen3.8 Thinking depth (low, medium default, or xhigh); ignored by other models
• quantization: Memory mode — 4-bit/8-bit/FP16 (HF only, ignored for GGUF)
• attention_mode: Attention backend — auto/manual selection (HF only, ignored for GGUF)
• preset_prompt: Pre-defined task prompts (❌ None sends no system instruction)
• custom_prompt: Custom text prompt (replaces preset when filled)
• max_tokens: Maximum new text tokens, including thinking and the final answer (64–256000). Input text and images count toward ctx separately; limited context can end generation earlier.
• keep_model_loaded: Cache model in VRAM between runs
• seed: Reproducibility seed

Generation Defaults (Basic node):

• Qwen3.5+ Thinking: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, repetition_penalty 1.0
• Qwen3.5+ Instruct: temperature 0.7, top_p 0.8, top_k 20, min_p 0.0, repetition_penalty 1.0
• Older models: temperature 0.6, top_p 0.9, repetition_penalty 1.2

Inputs:

• image (optional): Single image input
• video (optional): Video frames sequence
• source_path (optional): File path input

QwenVL (Advanced)

Full-featured node with granular control over all generation parameters.

Additional Parameters (beyond Basic):

• use_torch_compile: Enable torch.compile() JIT optimization (HF only, ignored for GGUF)
• device: Device override — auto/cuda:x/cpu
• temperature: Sampling randomness (0.0–2.0, default: 0.6). Values <0.01 trigger greedy decoding.
• top_p: Nucleus sampling threshold (0.0–1.0, default: 0.9)
• num_beams: Beam search width (1–8, default: 1). Values >1 disable temperature/top_p. (HF only, ignored for GGUF)
• repetition_penalty: Token repetition penalty (0.5–2.0, default: 1.2)
• frame_count: Video frame sampling count (1–64, default: 16)
• sampling_profile: Manual preserves the fields above; Qwen Recommended applies the model/mode preset and uses one sampling beam
• reasoning_effort: Qwen3.8 Thinking depth (low, medium default, or xhigh)

HF-specific optional parameters (ignored for GGUF):

• min_pixels: Minimum image resolution (default: 256×28×28 = 200,704)
• max_pixels: Maximum image resolution (default: 1280×28×28 = 1,003,520)

GGUF-specific optional parameters (ignored for HF):

• ctx: Context window size in tokens (0 = Auto for unified Qwen3.5+ models, range: 0–262144). Saved positive values remain explicit limits.
• n_batch: Prompt processing batch size (default: 512, range: 64–32768)
• gpu_layers: GPU layer offload count, -1 = all (default: -1)
• image_max_tokens: Max tokens per image encoding (default: 4096)
• top_k: Top-K for llama.cpp model constructor (default: 0 = disabled)
• min_p: Minimum probability sampling threshold (default: 0.0 = disabled, matches HF). Higher values (e.g. 0.05) narrow token candidates and may reduce output length.
• top_k_sampling: Top-K sampling during generation (default: 0 = disabled, matches HF). Non-zero values (e.g. 40) restrict each token choice to K most probable candidates.
• pool_size: Memory pool size for vision processing (default: 4,194,304)

Input Utility Nodes

Load Image Advanced

Loads images with additional outputs for mask and file path.

Returns:

• image: Image tensor
• mask: Alpha channel mask
• path: File path string

Features:

• Supports animated images (GIF)
• Auto EXIF orientation
• Multiple image formats (JPG, PNG, BMP, TIFF, WebP, GIF)

Load Video Advanced

Loads video files from ComfyUI input directory.

Returns:

• video: Video object
• path: File path string

Load Video Advanced (Path)

Loads video files from custom file path string.

Input:

• file: File path string (e.g., "X://path/to/video.mp4")

Returns:

• video: Video object
• path: File path string

Multiple Paths Input

Creates a path batch from multiple image/video files.

Parameters:

• inputcount: Number of input paths (1-1000)
• path_1, path_2, ...: Individual file paths
• sample_fps: Video sampling FPS (default: 1)
• max_frames: Maximum frames per video (default: 2)
• use_total_frames: Use all video frames (default: True)
• use_original_fps_as_sample_fps: Use original video FPS (default: True)

Returns:

• paths: List of path objects for batch processing

Supported Formats:

• Images: JPG, JPEG, PNG, BMP, TIFF, WebP, GIF
• Videos: MP4, MKV, MOV, AVI, FLV, WMV, WebM, M4V

Qwen3.5+ Unified Thinking/Instruct Mode

Qwen3.5, Qwen3.6, and Qwen3.8 models embed both reasoning and direct-response capabilities in a single unified model. Instead of choosing separate files, select the mode at inference time via the enable_thinking toggle:

Mode	enable_thinking	Behavior	Best For
Thinking	✅ ON	Model reasons step-by-step in <think>…</think> blocks before the final answer	Complex reasoning, math, code analysis, multi-step tasks
Instruct	❌ OFF (default)	Model responds directly without internal reasoning	Quick descriptions, tagging, straightforward Q&A

How It Works

• The enable_thinking parameter is passed to apply_chat_template(enable_thinking=True/False) for HF models
• For GGUF models, the toggle is passed to Qwen35 or Generic MTMD chat-template arguments and also controls output post-processing.
• When a model without the unified_thinking flag is selected, the toggle is ignored — behavior is unchanged for Qwen3-VL, Qwen2.5-VL, etc.

Sampling Recommendations

Mode	Temperature	top_p	top_k	presence_penalty
Qwen3.5 Thinking (general)	1.0	0.95	20	1.5
Qwen3.6/3.8 Thinking (general)	1.0	0.95	20	0.0
Instruct (general)	0.7	0.8	20	1.5

The Basic node applies these presets automatically. The Advanced node remains manual by default and applies them only when sampling_profile is set to Qwen Recommended.

Usage Guide

For High VRAM Systems (16GB+):

quantization: None (FP16)
attention_mode: flash_attention_2
keep_model_loaded: True
use_torch_compile: True

For Low VRAM Systems (<8GB):

quantization: 4-bit (VRAM-friendly)
attention_mode: auto
keep_model_loaded: False
Use GGUF models with Q4_K_M quantization

For Video Analysis:

frame_count: 16-32 (balance detail vs. speed)
max_tokens: 2048-4096 (longer outputs)

Preset Prompts

Prompt	Use Case	Output Type
❌ None	No system prompt	Custom only
🖼️ Tags	Generate comma-separated tags	Short list
🖼️ Simple Description	One-sentence summary	1 sentence
🖼️ Detailed Description	Comprehensive paragraph	6-10 sentences
🖼️ Ultra Detailed Description	Exhaustive analysis	10-16 sentences
🎬 Cinematic Description	Film-style description	Atmospheric
🖼️ Detailed Analysis	Structured breakdown	Categorized
📹 Video Summary	Video content summary	Narrative
📖 Short Story	Creative storytelling	Fiction
🪄 Prompt Refine & Expand	Enhance T2I prompts	Enhanced text

Attention Mode Selection

Auto Mode Priority

When attention_mode: auto, the system selects in this order:

1. Flash Attention 2 (flash_attention_2):

   • Best raw performance
   • Requires: Ampere+ GPU (RTX 30xx/40xx, A100, H100)
   • Install: pip install flash-attn --no-build-isolation

2. SDPA Flash (sdpa_flash):

   • PyTorch 2.0+ built-in Flash backend
   • Excellent performance with better compatibility (recommended)
   • Supports newer architectures (Blackwell, etc.)
   • Requires: Ampere+ GPU

3. SageAttention (sage_attention):

   • Memory efficient wrapper
   • Experimental feature
   • Install: pip install sageattention

4. SDPA Math (sdpa_math):

   • PyTorch SDPA with math backend
   • Slower but more stable fallback

5. Eager (eager):

   • Standard PyTorch attention
   • Always available (slowest)

Manual Selection

Force specific backend by setting attention_mode:

• auto: Auto-select best available (recommended)
• flash_attention_2: External flash-attn package
• sdpa_flash: PyTorch SDPA with Flash backend
• sdpa_math: PyTorch SDPA with math backend (disable Flash)
• sdpa: Legacy option (auto-selects Flash or math)
• sage_attention: SageAttention wrapper
• eager: Standard PyTorch attention

Performance Benchmarks

Inference Speed (approximate)

Model	GPU	Quantization	Tokens/sec	VRAM Usage
Qwen3-VL-4B	RTX 4090	FP16	~120	10GB
Qwen3-VL-4B	RTX 4090	8-bit	~100	6GB
Qwen3-VL-4B	RTX 4090	4-bit	~80	4GB
Qwen3-VL-8B	RTX 4090	8-bit	~70	10GB
Qwen3-VL-4B-GGUF	RTX 4090	Q4_K_M	~90	4GB

Benchmarks vary based on image resolution, prompt length, and hardware.

Troubleshooting

Common Issues

GGUF context limit reached:

max_tokens limits new text only; images, video frames, and prompt text also consume ctx. Basic and Advanced ctx=0 automatically size the context for unified Qwen3.5+ models and perform one exact preflight expansion when needed. A positive Advanced ctx remains an explicit limit. If the native context or an OOM fallback still reduces the available output budget, generation returns the available text and logs the requested and effective budgets.

"Out of Memory" Error:

• Solution 1: Use lower quantization (8-bit → 4-bit)
• Solution 2: Use GGUF models with Q4_K_M
• Solution 3: Disable keep_model_loaded
• Solution 4: Close other applications
• Solution 5: Use smaller model (8B → 4B → 2B)

"ImportError: transformers":

pip install "torch>=2.5.0" "transformers>=5.8.0"

Qwen3.5/Qwen3.6/Qwen3.8 fails to load (HF backend):

• trust_remote_code=True is handled automatically.
• Install torch >= 2.5.0 and transformers >= 5.8.0; the backend prefers AutoModelForMultimodalLM as its generic fallback.

Qwen3.5+ GGUF not loading:

• Requires JamePeng's llama-cpp-python fork >= 0.3.49.
• Qwen3.6 requires Qwen35ChatHandler; Qwen3.8 requires GenericMTMDChatHandler.
• Run python install.py with ComfyUI's Python from this custom node directory. The installer reports the detected platform and backend.

"llama-cpp-python not found" (GGUF):

cd ComfyUI/custom_nodes/ComfyUI-QwenVL-Utils
python install.py

If automatic compilation fails, install a C/C++ compiler plus the CUDA Toolkit or Vulkan SDK requested in the installer log, then rerun the same command. Apple Silicon Metal builds require the Xcode Command Line Tools. Available pre-built wheels are published on the JamePeng release page.

"Flash Attention not available":

• Requires Ampere or newer GPU (compute capability 8.0+)
• Install: pip install flash-attn --no-build-isolation
• Fallback: System will use SDPA automatically

Model Download Fails:

• Check internet connection
• Set HuggingFace token: huggingface-cli login
• Manual download from Hugging Face
• Place in ComfyUI/models/LLM/Qwen-VL/<model_name>/