Qwen3VL 🖤 NightShift

A ComfyUI custom node for multimodal inference with Qwen3VL vision-language models. Accepts text prompts, up to 4 images, and a video input in a single node — no extra helper nodes needed.

---

Features

• Single all-in-one node — model selection, prompting, generation settings, and image/video input in one place
• Up to 4 image inputs + video — pass individual images or an image batch as video frames
• Automatic image resizing — max_longer_edge widget resizes all inputs before inference to control vision token count and VRAM usage
• System prompt presets — dropdown populated from sysprompt_presets.json; select a preset or use a fully custom system prompt
• Flash Attention 2 support — automatically uses flash_attention_2 if installed, falls back to sdpa
• BF16 inference — native Qwen3VL dtype throughout; faster on Ampere+ GPUs (RTX 30xx/40xx)
• Smart caching — model, processor, and tokenizer are cached after first load; IS_CHANGED hashing skips re-execution when nothing has changed
• Cancel support — responds to ComfyUI's "Cancel current run" button during generation
• Progress bar — token count and tok/s logged to console during generation
• Detailed console logging — timing stats, VRAM usage, vision token estimates, and resize info on every run

---

Installation

Manual:

1. Clone or download this repo into your ComfyUI/custom_nodes/ folder:

   cd ComfyUI/custom_nodes
   git clone https://github.com/SmokingDormouse/ComfyUI-Qwen3VL-NightShift.git
   

2. Restart ComfyUI.

---

Model Setup

Place your Qwen3VL model folder inside:

ComfyUI/models/llm/Qwen-VL/

Each model should be a subfolder containing a config.json. Example:

ComfyUI/models/llm/Qwen-VL/Qwen3-VL-4B-Instruct/

The node scans this folder at startup and populates the model_name dropdown automatically.

Recommended model: Huihui-AI/Huihui-Qwen3-VL-4B-Instruct-abliterated

---

Widgets

Widget	Description
max_longer_edge	Resize all images/frames so the longer edge equals this value before inference. Set to 0 to disable. Default: 768
model_name	Qwen3VL model folder to load
preset_system_prompt	Select a preset system prompt from sysprompt_presets.json, or — Custom — to use the system_prompt widget
user_prompt	The user message sent to the model
system_prompt	Custom system prompt (used when preset is set to — Custom —)
max_length	Maximum number of new tokens to generate
temperature	Sampling temperature. Set to 0 for greedy/deterministic output
top_p	Nucleus sampling probability
num_beams	Beam search width. 1 = greedy/sampling, >1 = beam search
repetition_penalty	Penalizes repeated tokens
seed	Generation seed. Use control_after_generate to fix or randomize
tf32	Enable TF32 on CUDA for faster matrix ops on Ampere+ GPUs
attention	Attention implementation: Auto, flash_attention_2, sdpa, or eager
torch_compile	Enable torch.compile for potential speedup after warmup
keep_model_loaded	Keep model in VRAM between runs. Disable to free VRAM after each run
video_frames	Number of frames to sample from the video input (min: 2)
device	nvidia_gpu or cpu

Optional inputs: 🖼️ Image 1–4, 🎥 Video (IMAGE batch)

---

System Prompt Presets

Presets are loaded from sysprompt_presets.json in the node folder at startup. The file uses a simple flat structure:

{
  "presets": [
    {
      "name": "My Preset",
      "system_prompt": "You are a..."
    }
  ]
}

Add, edit, or remove presets by editing this file and restarting ComfyUI. The preset dropdown is built from this file — no code changes needed.

When a preset is selected, the system_prompt widget is ignored. The user_prompt widget is always used regardless of preset selection.

---

Optional: Flash Attention 2

For significantly faster inference, install flash-attn if your environment supports it. The node detects it automatically at startup and switches to flash_attention_2. If not installed, it falls back to PyTorch SDPA transparently.

---

Debug Mode

Set the environment variable NIGHTSHIFT_DEBUG=1 before starting ComfyUI to enable detailed CUDA memory summaries on errors.