diff --git a/README.md b/README.md index f382976..521dea9 100644 --- a/README.md +++ b/README.md @@ -7,12 +7,14 @@ Navigate to the root ComfyUI directory, clone the repository to `custom_nodes` a git clone https://github.com/Zuellni/ComfyUI-ExLlama-Nodes custom_nodes/ComfyUI-ExLlama-Nodes pip install -r custom_nodes/ComfyUI-ExLlama-Nodes/requirements.txt ``` -Optionally, you can install [flash-attention](https://github.com/Dao-AILab/flash-attention) by uncommenting the relevant lines in the requirements file. It should lower VRAM usage but your mileage may vary. -> [!IMPORTANT] -> The wheels included in the requirements file should match the latest portable ComfyUI build. If you see any ExLlama-related errors while loading the nodes, try to install it manually following the [official instructions](https://github.com/turboderp/exllamav2#installation). +Optionally, you can install [flash-attention](https://github.com/Dao-AILab/flash-attention) by uncommenting relevant lines in the requirements file. It should lower VRAM usage but your mileage may vary. +> [!CAUTION] +> The wheels included in the requirements file should match the `stable pytorch 2.1 cu121` build of ComfyUI.
+> If you see any ExLlama-related errors while loading the nodes, try to install it following the [official instructions](https://github.com/turboderp/exllamav2#installation).
+> Keep in mind that wheels >= `0.0.13` require `pytorch 2.2`. ## Usage -Only EXL2 and 4-bit GPTQ models are supported. You can find a lot of them on [Hugging Face](https://huggingface.co/TheBloke). Refer to the model card in each repository for details about quant differences and instruction formats. +Only EXL2 and 4-bit GPTQ models are supported. You can find a lot of them on [Hugging](https://huggingface.co/LoneStriker) [Face](https://huggingface.co/TheBloke). Refer to the model card in each repository for details about quant differences and instruction formats. To use a model with the nodes, you should clone its repository with git or manually download all the files and place them in `models/llm`. For example, if you'd like to download [Mistral-7B](https://huggingface.co/LoneStriker/Mistral-7B-Instruct-v0.2-5.0bpw-h6-exl2-2), use the following command: ```