Update README.md
This commit is contained in:
@@ -6,11 +6,13 @@
|
||||
|
||||
## Features
|
||||
|
||||
- Supports MiniCPM models including **MiniCPM-V-4 (GGUF)**.
|
||||
- Multiple caption types to suit different use cases (Describe, Caption, Analyze, etc.).
|
||||
- Memory management options to balance VRAM usage and speed.
|
||||
- Auto-downloads model files on first use for easy setup.
|
||||
- Customizable parameters: max tokens, temperature, top-p/k sampling, repetition penalty.
|
||||
- Supports both **MiniCPM-V-4 (Transformers)** and **MiniCPM-V-4 (GGUF)** models
|
||||
- Multiple caption types to suit different use cases (Describe, Caption, Analyze, etc.)
|
||||
- Memory management options to balance VRAM usage and speed
|
||||
- Auto-downloads model files on first use for easy setup
|
||||
- Customizable parameters: max tokens, temperature, top-p/k sampling, repetition penalty
|
||||
- Advanced node with full parameter control
|
||||
- Legacy node for backward compatibility
|
||||
|
||||
---
|
||||
|
||||
@@ -21,7 +23,7 @@ Clone the repo into your ComfyUI custom nodes folder:
|
||||
```bash
|
||||
cd ComfyUI/custom_nodes
|
||||
git clone https://github.com/1038lab/comfyui-minicpm.git
|
||||
````
|
||||
```
|
||||
|
||||
Install required dependencies:
|
||||
|
||||
@@ -33,20 +35,55 @@ pip install -r ComfyUI/custom_nodes/comfyui-minicpm/requirements.txt
|
||||
|
||||
## Supported Models
|
||||
|
||||
### Transformers Models
|
||||
| Model | Description |
|
||||
| ------------------ | -------------------------------------- |
|
||||
| MiniCPM-V-4-int4 | 4-bit quantized version, smaller memory footprint |
|
||||
| MiniCPM-V-4 | Full precision version, higher quality |
|
||||
|
||||
### GGUF Models
|
||||
| Model | Description |
|
||||
| ------------------ | -------------------------------------- |
|
||||
| MiniCPM-V-4 (GGUF) | Latest stable GGUF model, best quality |
|
||||
|
||||
> The models will be automatically downloaded on first run.
|
||||
> Manual download and placement into `models/LLM/GGUF` is also supported.
|
||||
> Manual download and placement into `models/prompt_generator` (transformers) or `models/LLM/GGUF` (GGUF) is also supported.
|
||||
|
||||
---
|
||||
|
||||
## Available Nodes
|
||||
|
||||
### 1. MiniCPM-4-V-Transformers
|
||||
- Basic transformers-based node with essential parameters
|
||||
- Supports image and video input
|
||||
- Memory management options
|
||||
- Preset prompt types
|
||||
|
||||
### 2. MiniCPM-4-V-Transformers Advanced
|
||||
- Full-featured transformers-based node
|
||||
- All parameters customizable
|
||||
- System prompt support
|
||||
- Advanced video processing options
|
||||
|
||||
### 3. MiniCPM-4-V-GGUF
|
||||
- GGUF-based node with essential parameters
|
||||
- Optimized for performance
|
||||
|
||||
### 4. MiniCPM-4-V-GGUF Advanced
|
||||
- Full-featured GGUF-based node
|
||||
- All parameters customizable
|
||||
|
||||
### 5. MiniCPM (Legacy)
|
||||
- Original node for backward compatibility
|
||||
- Basic functionality
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
1. Add the **MiniCPM** node from the `🧪AILab` category in ComfyUI.
|
||||
2. Connect an image input node to the MiniCPM node.
|
||||
3. Select the model variant (default is MiniCPM-V-4 GGUF).
|
||||
2. Connect an image or video input node to the MiniCPM node.
|
||||
3. Select the model variant (default is MiniCPM-V-4-int4 for transformers).
|
||||
4. Choose caption type and adjust parameters as needed.
|
||||
5. Execute your workflow to generate captions or analysis.
|
||||
|
||||
@@ -60,11 +97,11 @@ pip install -r ComfyUI/custom_nodes/comfyui-minicpm/requirements.txt
|
||||
"gpu_layers": -1,
|
||||
"cpu_threads": 4,
|
||||
"default_max_tokens": 1024,
|
||||
"default_temperature": 0.8,
|
||||
"default_temperature": 0.7,
|
||||
"default_top_p": 0.9,
|
||||
"default_top_k": 80,
|
||||
"default_repetition_penalty": 1.05,
|
||||
"default_system_prompt": "You are MiniCPM, a helpful, concise and knowledgeable vision-language assistant. Answer directly and stay on task."
|
||||
"default_top_k": 100,
|
||||
"default_repetition_penalty": 1.10,
|
||||
"default_system_prompt": "You are MiniCPM-V, a helpful, concise and knowledgeable vision-language assistant. Answer directly and stay on task."
|
||||
}
|
||||
```
|
||||
|
||||
@@ -80,6 +117,20 @@ pip install -r ComfyUI/custom_nodes/comfyui-minicpm/requirements.txt
|
||||
* **List:** List the main objects visible in this image.
|
||||
* **Scene:** Describe the scene and setting of this image.
|
||||
* **Details:** What are the key details in this image?
|
||||
* **Summarize:** Summarize the key content of this image in 1-2 sentences.
|
||||
* **Emotion:** Describe the emotions or mood conveyed by this image.
|
||||
* **Style:** Describe the artistic or visual style of this image.
|
||||
* **Location:** Where might this image be taken? Analyze the setting or location.
|
||||
* **Question:** What question could be asked based on this image?
|
||||
* **Creative:** Describe this image as if writing the beginning of a short story.
|
||||
|
||||
---
|
||||
|
||||
## Memory Management Options
|
||||
|
||||
* **Keep in Memory:** Model stays loaded for faster subsequent runs
|
||||
* **Clear After Run:** Model is unloaded after each run to save memory
|
||||
* **Global Cache:** Model is cached globally and shared between nodes
|
||||
|
||||
---
|
||||
|
||||
@@ -89,6 +140,8 @@ pip install -r ComfyUI/custom_nodes/comfyui-minicpm/requirements.txt
|
||||
* Use top-p (0.9) and top-k (80) sampling for natural output diversity.
|
||||
* Lower max tokens or precision (bf16/fp16) for faster generation on less powerful GPUs.
|
||||
* Memory modes help optimize VRAM usage: default, balanced, max savings.
|
||||
* Transformers models offer better quality but use more memory.
|
||||
* GGUF models are more memory-efficient but may have slightly lower quality.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user