100 lines
3.6 KiB
Markdown
100 lines
3.6 KiB
Markdown
# 🧩 ComfyUI-QwenVL — Custom Models Guide
|
|
|
|
You can add your own custom HuggingFace or GGUF fine-tuned models without modifying the built-in configuration files.
|
|
|
|
---
|
|
|
|
## 📁 File Location
|
|
Place a file named `custom_models.json` in the plugin root folder:
|
|
|
|
```
|
|
ComfyUI/custom_nodes/ComfyUI-QwenVL/custom_models.json
|
|
```
|
|
|
|
When ComfyUI starts (or when the Downloader runs), it automatically detects and merges this file with the default model catalogs (`hf_models.json` and `gguf_models.json`).
|
|
|
|
---
|
|
|
|
## ⚙️ Simplified File Format
|
|
All custom models are organized into just two clear sections:
|
|
|
|
```json
|
|
{
|
|
"hf_models": {
|
|
"My-Custom-Qwen2.5-VL-7B": {
|
|
"repo_id": "myusername/Qwen2.5-VL-7B-Finetune",
|
|
"default": false,
|
|
"quantized": false,
|
|
"vram_requirement": {
|
|
"full": 15.0,
|
|
"8bit": 8.5,
|
|
"4bit": 5.0
|
|
}
|
|
}
|
|
},
|
|
"gguf_models": {
|
|
"Huihui-Qwen3.5-4B-abliterated-GGUF": {
|
|
"author": "mradermacher",
|
|
"repo_name": "Huihui-Qwen3.5-4B-abliterated-GGUF",
|
|
"repo_id": "mradermacher/Huihui-Qwen3.5-4B-abliterated-GGUF",
|
|
"mmproj_file": "Huihui-Qwen3.5-4B-abliterated.mmproj-f16.gguf",
|
|
"model_files": [
|
|
"Huihui-Qwen3.5-4B-abliterated.Q4_K_M.gguf",
|
|
"Huihui-Qwen3.5-4B-abliterated.Q8_0.gguf"
|
|
],
|
|
"defaults": {
|
|
"context_length": 8192
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Sections:
|
|
- **`hf_models`**: Transformers / PyTorch models (for `AILab_QwenVL`, `AILab_QwenVL_Advanced`, and `AILab_QwenVL_PromptEnhancer`).
|
|
- **`gguf_models`**: GGUF format models (for `AILab_QwenVL_GGUF`, `AILab_QwenVL_GGUF_Advanced`, and `AILab_QwenVL_GGUF_PromptEnhancer`).
|
|
- `mmproj_file`: (Optional) The vision projector file. If provided, vision nodes use it for visual understanding. Text-only nodes (like Prompt Enhancer) simply ignore it.
|
|
|
|
---
|
|
|
|
## 💾 Using Models from Another Drive or Local Directory
|
|
|
|
If you already have models downloaded on your disk (or locally fine-tuned models) and want to use them **without downloading from HuggingFace**, follow ComfyUI's standard model directory architecture:
|
|
|
|
### Method 1: Via ComfyUI's `extra_model_paths.yaml` (Recommended)
|
|
In your `ComfyUI/extra_model_paths.yaml`, configure your external drive or model directory:
|
|
|
|
```yaml
|
|
my_other_drive:
|
|
base_path: D:/models/
|
|
llm: LLM
|
|
```
|
|
Place your models inside:
|
|
- **HF Models**: `D:/models/LLM/<repo_name>/` (containing `config.json` & weights)
|
|
- **GGUF Models**: `D:/models/LLM/GGUF/<repo_name>/` (or directly inside `D:/models/LLM/`)
|
|
|
|
Register the model in `custom_models.json` using its matching `repo_id` (e.g. `"repo_id": "author/repo_name"`). The nodes automatically scan all paths registered in `extra_model_paths.yaml` (both uppercase `LLM` and lowercase `llm`), load the local weights, and skip downloading.
|
|
|
|
### Method 2: Standard ComfyUI Models Folder
|
|
Place your model folder into:
|
|
```
|
|
ComfyUI/models/LLM/<repo_name>/
|
|
```
|
|
As long as `*.safetensors` or `*.bin` files exist locally, the node loads from disk directly and skips downloading.
|
|
|
|
### Method 3: Directory Junction / Symlink (Recommended for Zero-Copy)
|
|
To seamlessly share models across drives without duplicating files or reconfiguring:
|
|
- **Windows** (run in Command Prompt or PowerShell):
|
|
```cmd
|
|
mklink /J "path\to\ComfyUI\models\LLM" "D:\YourModels\LLM"
|
|
```
|
|
- **Linux / macOS**:
|
|
```bash
|
|
ln -s /path/to/other_drive/LLM path/to/ComfyUI/models/LLM
|
|
```
|
|
|
|
---
|
|
|
|
## 📥 Automatic Registration via Downloader
|
|
Using the **QwenVL HuggingFace Downloader 📥** node (`AILab_HuggingFaceDownloader`), any downloaded model will be automatically created and registered into `custom_models.json` with all correct settings and matching `mmproj` files!
|