Update dependencies
This commit is contained in:
Level Pixel Dev
@@ -5,13 +5,12 @@
|
||||
The purpose of this package is to collect the most necessary and atomic nodes for working with LLM and VLM models with GGUF format. Installation and maintenance of LLM and VLM nodes based on LLaVA is more complex, so this node package should now be installed separately from the main Level Pixel node package.
|
||||
|
||||
**In this Level Pixel Advanced node pack you will find:**
|
||||
LLM nodes, LLaVa and other VLM nodes
|
||||
LLM nodes, LLaVa and other VLM nodes, Autotagger, RAM autotagger, WD Autotagger
|
||||
|
||||
Recommend that you install the main node package from Level Pixel:\
|
||||
Recommend that you install the main node package from Level Pixel:
|
||||
[https://github.com/LevelPixel/ComfyUI-LevelPixel](https://github.com/LevelPixel/ComfyUI-LevelPixel)
|
||||
|
||||
|
||||
The official repository of the current node package is located at this link:\
|
||||
The official repository of the current node package is located at this link:
|
||||
[https://github.com/LevelPixel/ComfyUI-LevelPixel-Advanced](https://github.com/LevelPixel/ComfyUI-LevelPixel-Advanced)
|
||||
|
||||
## Contacts:
|
||||
@@ -19,22 +18,21 @@ The official repository of the current node package is located at this link:\
|
||||
For cooperation, suggestions and ideas you can write to email:
|
||||
levelpixel.dev@gmail.com
|
||||
|
||||
|
||||
# Installation:
|
||||
|
||||
### Pre-Installation:
|
||||
|
||||
Before running ComfyUI with this node package, you should make sure that you have the following programs and libraries installed so that ComfyUI can compile the necessary libraries and programs for llama-cpp-python (the main library that allows you to use any current gguf models and neural network architectures):
|
||||
Before running ComfyUI with this node package, you should make sure that you have the following programs and libraries installed so that ComfyUI can compile the necessary libraries and programs for llama-cpp-python (the main library that allows you to use any current GGUF models and neural network architectures):
|
||||
|
||||
* `CUDA drivers` - install the latest version.\
|
||||
* `CUDA drivers` - install the latest version.
|
||||
Download: [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads)
|
||||
* `Visual Studio 2022 Community IDE` with libraries for compiling C++ programs, specifically with individual components (select them in Visual Studio Installer when installing/modifying Visual Studio 2022):\
|
||||
MSVC v143 - VS 2022 C++ x64/x86 build tools (Latest)\
|
||||
MSVC v143 - VS 2022 C++ x64/x86 build tools (v14.38-17.8)\
|
||||
C++ Cmake tools for Windows\
|
||||
C++ Cmake tools for Linux and Mac\
|
||||
* `Visual Studio 2022 Community IDE` with libraries for compiling C++ programs, specifically with individual components (select them in Visual Studio Installer when installing/modifying Visual Studio 2022):
|
||||
MSVC v143 - VS 2022 C++ x64/x86 build tools (Latest)
|
||||
MSVC v143 - VS 2022 C++ x64/x86 build tools (v14.38-17.8)
|
||||
C++ Cmake tools for Windows
|
||||
C++ Cmake tools for Linux and Mac
|
||||
Download: [https://visualstudio.microsoft.com/downloads](https://visualstudio.microsoft.com/downloads)
|
||||
* `CMAKE official distribution` - download and install the latest version.\
|
||||
* `CMAKE official distribution` - download and install the latest version.
|
||||
Download: [https://cmake.org/download](https://cmake.org/download)
|
||||
|
||||
### Installation Using ComfyUI Manager (recommended):
|
||||
@@ -65,10 +63,10 @@ If you have problems running ComfyUI with this node package, check and do the fo
|
||||
* The path to Cmake (official distribution, for example, path `C:\Program Files\CMake\bin`) must be at the very top of the Path list in the **System environment variables**. Make sure that the path is at the top of the Path in the **System variables**, and not only in the user variables. If Cmake is not at the top of the list, then some libraries may not compile due to the lack of the current version of the Cmake compiler.
|
||||
* The path to the current version of your CUDA must be at the very top of the **Path** list in the **System environment variables** (right after Cmake). Make sure that the path is at the top of the Path in the System variables, and not only in the user variables.
|
||||
Here are the paths that should be in Path for CUDA (this is an example, substitute your CUDA version for "12.9"):
|
||||
|
||||
`C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\libnvvp`\
|
||||
|
||||
`C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\libnvvp`
|
||||
`C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin`
|
||||
* In System environment variables, add the `CMAKE_ARGS` variable and set it to the following:\
|
||||
* In System environment variables, add the `CMAKE_ARGS` variable and set it to the following:
|
||||
`-DGGML_CUDA=on -DCMAKE_GENERATOR_TOOLSET='cuda=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9'`
|
||||
|
||||
Save the changes made to System environment variables.
|
||||
@@ -107,34 +105,61 @@ The core functionality is taken from [ComfyUI_VLM_nodes](https://github.com/goka
|
||||
|
||||
How to use LLaVa nodes:
|
||||
|
||||
1. **Download the Qwen 2.5 VL gguf file**:\
|
||||
1. **Download the Qwen 2.5 VL gguf file**:
|
||||
[https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main)\
|
||||
Choose a gguf file **without** the mmproj in the name\
|
||||
Example gguf file:\
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main)
|
||||
Choose a gguf file **without** the mmproj in the name
|
||||
Example gguf file:
|
||||
[https://huggingface.co/Mungert/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-q8_0.gguf
|
||||
](https://huggingface.co/Mungert/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-q8_0.gguf)\
|
||||
](https://huggingface.co/Mungert/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-q8_0.gguf)
|
||||
Copy this file to ComfyUI/model/Llavacheckpoint.
|
||||
|
||||
2. **Download the Qwen 2.5 VL mmproj file (this is clip model):**\
|
||||
2. **Download the Qwen 2.5 VL mmproj file (this is clip model):**
|
||||
[https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main)\
|
||||
Choose a file **with** mmproj in the name\
|
||||
Example mmproj file:\
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/tree/main)
|
||||
Choose a file **with** mmproj in the name
|
||||
Example mmproj file:
|
||||
[https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-mmproj-bf16.gguf
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-mmproj-bf16.gguf)\
|
||||
](https://huggingface.co/Mungert/Qwen2.5-VL-7B-Instruct-GGUF/resolve/main/Qwen2.5-VL-7B-Instruct-mmproj-bf16.gguf)
|
||||
Copy this file to ComfyUI/model/Llavacheckpoint.
|
||||
|
||||
3. **Run ComfyUI and add to workflow node LLava Advanced [LP]. Choose ckpt model and clip, pin image and write prompt.**
|
||||
|
||||
## Autotagger nodes based on WD tagger models
|
||||
|
||||
An image autotagger that creates highly relevant tags using fast and ultra-accurate, highly specialized models. More diverse models are planned to be added to the list of models in the future.
|
||||
|
||||
This node allows it to be used in cycles and conditions (in places where it is not necessary to execute this node according to the specified conditions), since it is not a node with mandatory execution.
|
||||
|
||||
The core functionality is taken from [ComfyUI-WD14-Tagger](https://github.com/pythongosssss/ComfyUI-WD14-Tagger) and belongs to its authors.
|
||||
|
||||
## Recognize Anything Model (RAM++)
|
||||
|
||||
The counterpart to Segment Anything Model (SAM)
|
||||
|
||||
This is an image recognition node for [ComfyUI](https://github.com/comfyanonymous/ComfyUI) based on the RAM++ model from [xinyu1205](https://huggingface.co/xinyu1205).
|
||||
|
||||
* [https://huggingface.co/xinyu1205/recognize-anything-plus-model](https://huggingface.co/xinyu1205/recognize-anything-plus-model)
|
||||
* [https://github.com/xinyu1205/recognize-anything](https://github.com/xinyu1205/recognize-anything)
|
||||
* This node outputs a string of tags with all the recognized objects and elements in the image
|
||||
* 3 different models.
|
||||
* RAM and RAM++ outputs tags in English and Chinese language
|
||||
|
||||
Furthermore you need to download the [RAM](https://huggingface.co/xinyu1205/recognize_anything_model/resolve/main/ram_swin_large_14m.pth), [RAM++](https://huggingface.co/xinyu1205/recognize-anything-plus-model/resolve/main/ram_plus_swin_large_14m.pth) and [tag2text](https://huggingface.co/xinyu1205/recognize_anything_model/resolve/main/tag2text_swin_14m.pth) models and place it in the /ComfyUI/models/rams/ folder or use the [ComfyUI-Manager](https://github.com/Comfy-Org/ComfyUI-Manager) model downloader.
|
||||
|
||||
You can also configure the location in 'extra\_model\_paths.yaml' in the Confyui folder.
|
||||
|
||||
# Credits
|
||||
|
||||
ComfyUI/[ComfyUI](https://github.com/comfyanonymous/ComfyUI) - A powerful and modular stable diffusion GUI.
|
||||
|
||||
VLM nodes for ComfyUI/[ComfyUI_VLM_nodes](https://github.com/gokayfem/ComfyUI_VLM_nodes) - Best VLM nodes for ComfyUI.
|
||||
|
||||
WD autotagger node for ComfyUI/[ComfyUI-WD14-Tagger](https://github.com/pythongosssss/ComfyUI-WD14-Tagger) - Source ComfyUI nodes for WD autotagger
|
||||
|
||||
RAM node for ComfyUI/[ComfyUI-Hangover-Recognize_Anything](https://github.com/Hangover3832/ComfyUI-Hangover-Recognize_Anything) - Source ComfyUI nodes for RAM (source repository is archived, but we will continue to support RAM nodes)
|
||||
|
||||
# License
|
||||
|
||||
Copyright (c) 2024-present [Level Pixel](https://github.com/LevelPixel)
|
||||
|
||||
Licensed under Apache License
|
||||
|
||||
|
||||
+3
-1
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"Autotagger [LP]":"Autotagger based on the latest version of the WD tagging model from SmilingWolf.",
|
||||
"LLM Loader [LP]":"Loads a large language model specified by the user. Models are stored in the LLavacheckpoints folder.",
|
||||
"LLM Sampler [LP]":"Generates text using the LLM. Simple settings.",
|
||||
"LLM Advanced [LP]":"Generates text using the LLM. Advanced settings.",
|
||||
@@ -7,5 +8,6 @@
|
||||
"LLava Sampler Simple [LP]":"Generates text using the VLM. Simple settings.",
|
||||
"LLava Sampler Advanced [LP]":"Generates text using the VLM. Advanced settings.",
|
||||
"LLava Simple [LP]":"Generates text using the loaded VLM from LLava Loader. Simple settings.",
|
||||
"LLava Advanced [LP]":"Generates text using the loaded VLM from LLava Loader. Advanced settings."
|
||||
"LLava Advanced [LP]":"Generates text using the loaded VLM from LLava Loader. Advanced settings.",
|
||||
"Recognize Anything (RAM) [LP]":"Recognize Anything (RAM) - node for generating tags for an image using the Recognize Anything Plus Model"
|
||||
}
|
||||
@@ -20,7 +20,6 @@
|
||||
"wd-v1-4-convnext-tagger": "{HF_ENDPOINT}/SmilingWolf/wd-v1-4-convnext-tagger",
|
||||
"wd-v1-4-vit-tagger-v2": "{HF_ENDPOINT}/SmilingWolf/wd-v1-4-vit-tagger-v2",
|
||||
"wd-v1-4-swinv2-tagger-v2": "{HF_ENDPOINT}/SmilingWolf/wd-v1-4-swinv2-tagger-v2",
|
||||
"wd-v1-4-vit-tagger": "{HF_ENDPOINT}/SmilingWolf/wd-v1-4-vit-tagger",
|
||||
"Z3D-E621-Convnext": "{HF_ENDPOINT}/silveroxides/Z3D-E621-Convnext"
|
||||
"wd-v1-4-vit-tagger": "{HF_ENDPOINT}/SmilingWolf/wd-v1-4-vit-tagger"
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -105,9 +105,9 @@ class RecognizeAnything:
|
||||
return(tag0, tag1, tag2,)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Recognize Anything|LP": RecognizeAnything,
|
||||
"RecognizeAnything(RAM)|LP": RecognizeAnything,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Recognize Anything|LP": "Recognize Anything [LP]",
|
||||
"RecognizeAnything(RAM)|LP": "Recognize Anything (RAM) [LP]",
|
||||
}
|
||||
+1
-1
@@ -3,7 +3,7 @@ name = "comfyui-level-pixel-advanced"
|
||||
description = "Advanced nodes of the Level Pixel company. Includes convenient advanced nodes for working with LLM и VLM models (LLaVa) with GGUF format. Qwen2.5-VL and Qwen2.5 supported. Also included is a node for the RAM model. Nodes have the ability to automatically unload models from VRAM."
|
||||
version = "1.2.0"
|
||||
license = { file = "LICENSE" }
|
||||
dependencies = ["torch>=2.0.1", "torchvision>=0.15.2", "transformers>=4.46", "pillow>=9.4.0", "numpy>=1.26.4", "matplotlib", "scikit-build-core>=0.10.7", "onnxruntime-gpu>=1.20.0", "onnxruntime>=1.20.0"]
|
||||
dependencies = ["torch>=2.0.1", "torchvision>=0.15.2", "transformers>=4.46", "pillow>=9.4.0", "numpy>=1.26.4", "matplotlib", "scikit-build-core>=0.10.7", "onnxruntime-gpu>=1.20.0", "onnxruntime>=1.20.0", "timm>=0.4.12", "fairscale>=0.4.4"]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/LevelPixel/ComfyUI-LevelPixel-Advanced"
|
||||
|
||||
@@ -6,4 +6,6 @@ matplotlib
|
||||
scikit-build-core>=0.10.7
|
||||
onnxruntime-gpu>=1.20.0
|
||||
onnxruntime>=1.20.0
|
||||
timm>=0.4.12
|
||||
fairscale>=0.4.4
|
||||
ram @ git+https://github.com/xinyu1205/recognize-anything.git
|
||||
|
||||
Reference in New Issue
Block a user