nodes/Zuellni-ComfyUI-ExLlama-Nodes
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rename the lora loader node

Browse Source
This commit is contained in:
Zuellni
2023-09-24 09:59:08 +02:00
parent f85c061c4d
commit 3e4428d83c
4 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -18,7 +18,7 @@ python -m pip install https://github.com/jllllll/exllama/releases/download/0.0.1
Name | Description
:--- | :---
Loader | Used to load 4-bit GPTQ Llama/2 models. You can find a lot of them on [Hugging Face](https://huggingface.co/TheBloke).<br>Clone the model repository or download all the files and place them in an empty directory, then specify the path in `model_dir`. The `model.safetensors` file won't work on its own.<br><br>ExLlama allocates memory based on `max_seq_len`. Lowering it is a good way to save on VRAM.<br>It's currently not possible to [offload](https://github.com/turboderp/exllama/issues/177) the model to RAM.
LoRA Loader | Used to load LoRAs. The directory should contain `adapter_model.bin` and `adapter_config.json`.<br>LoRA parameter count has to match the model.
LoRA | Used to load LoRAs. The directory should contain `adapter_model.bin` and `adapter_config.json`.<br>LoRA parameter count has to match the model.
Generator | Generates a `string` based on the given `prompt` for use with other nodes.<br>Default values correspond to the `simple-1` preset from [text-generation-webui](https://github.com/oobabooga/text-generation-webui).<br><br>ExLlama isn't [deterministic](https://github.com/turboderp/exllama/issues/201), so the outputs may differ even with the same seed.
Previewer | Displays generated outputs in the UI and appends them to workflow metadata.
__init__.py
+3 -3
View File
@@ -1,15 +1,15 @@
from .nodes import Generator, Loader, LoraLoader, Previewer
from .nodes import Generator, Loader, Lora, Previewer
NODE_CLASS_MAPPINGS = {
"ZuellniExLlamaLoader": Loader,
"ZuellniExLlamaLoraLoader": LoraLoader,
"ZuellniExLlamaLora": Lora,
"ZuellniExLlamaGenerator": Generator,
"ZuellniExLlamaPreviewer": Previewer,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"ZuellniExLlamaLoader": "ExLlama Loader",
"ZuellniExLlamaLoraLoader": "ExLlama LoRA Loader",
"ZuellniExLlamaLora": "ExLlama LoRA",
"ZuellniExLlamaGenerator": "ExLlama Generator",
"ZuellniExLlamaPreviewer": "ExLlama Previewer",
}
nodes.py
+1 -1
View File
@@ -133,7 +133,7 @@ class Loader:
return (generator,)
class LoraLoader:
class Lora:
@classmethod
def INPUT_TYPES(cls):
return {
requirements.txt
+1 -1
View File
@@ -2,5 +2,5 @@ https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu121
https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu118-cp310-cp310-win_amd64.whl; platform_system == "Windows" and python_version == "3.10"
https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu121-cp311-cp311-linux_x86_64.whl; platform_system == "Linux" and python_version == "3.11"
https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu118-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and python_version == "3.10"
colorama
sentencepiece
colorama