85 lines
2.5 KiB
Markdown
85 lines
2.5 KiB
Markdown
# `llama-cpp-python` CUDA Installation for ComfyUI Portable
|
||
|
||
This guide helps you install `llama-cpp-python` with GPU (CUDA) support for ComfyUI's Windows portable version.
|
||
|
||
---
|
||
|
||
### **1. Open ComfyUI's Command Prompt**
|
||
|
||
* Go to `C:\ComfyUI_windows_portable\python_embeded`
|
||
* In the address bar, type `cmd` and press Enter.
|
||
|
||
---
|
||
|
||
### **2. Install Build Tools (If you haven't already)**
|
||
|
||
* **Install Visual Studio Build Tools:**
|
||
* Download Visual Studio Build Tools 2022 [https://visualstudio.microsoft.com/downloads/](https://visualstudio.microsoft.com/downloads/](https://learn.microsoft.com/en-us/visualstudio/releases/2022/release-history#release-dates-and-build-numbers) (under 17.4 "Build Tools").
|
||
* Run installer, select **"Desktop development with C++"** workload.
|
||
* **Install NVIDIA CUDA Toolkit:**
|
||
* Download your CUDA version (e.g., 12.6) from [https://developer.nvidia.com/cuda-toolkit-archive](https://developer.nvidia.com/cuda-toolkit-archive).
|
||
* Run installer, ensure **"Visual Studio Integration"** is selected.
|
||
|
||
## **Automatic Installation Script (Recommended)**
|
||
|
||
* Make sure the installation script is saved here:
|
||
`.\ComfyUI\custom_nodes\ComfyUI-JoyCaption\install_llama_official.py`
|
||
|
||
* In the command prompt (opened in step 1), run the script with:
|
||
|
||
```bash
|
||
.\python_embeded\python.exe llama_cpp_install.py
|
||
```
|
||
|
||
The script will:
|
||
|
||
* Upgrade pip
|
||
* Clear pip cache
|
||
* Install minimal build dependencies (scikit-build-core, cmake)
|
||
* Detect GPU and build llama-cpp-python with CUDA support if available
|
||
|
||
🕒 This process may take 5–20+ minutes depending on your system.
|
||
|
||
### ** Restart ComfyUI**
|
||
|
||
* Close ComfyUI.
|
||
* Restart ComfyUI.
|
||
* Hard refresh your browser with Ctrl+F5 (Windows) or Cmd+Shift+R (macOS).
|
||
|
||
|
||
|
||
|
||
## **Manuelly Installation**
|
||
---
|
||
|
||
### **1. Prepare Python Environment**
|
||
|
||
* In the command prompt, run:
|
||
```bash
|
||
.\python.exe -m pip install --upgrade pip
|
||
.\python.exe -m pip cache purge
|
||
.\python.exe -m pip install scikit-build-core cmake
|
||
```
|
||
|
||
---
|
||
|
||
### **2. Compile and Install `llama-cpp-python`**
|
||
|
||
* In the command prompt, run:
|
||
```bash
|
||
set CMAKE_ARGS="-DGGML_CUDA=on" && .\python.exe -m pip install llama-cpp-python --no-cache-dir && set CMAKE_ARGS=
|
||
```
|
||
* **Wait patiently.** This takes time (5-20+ minutes).
|
||
|
||
---
|
||
|
||
### **3. Restart ComfyUI**
|
||
|
||
* Close ComfyUI.
|
||
* Restart ComfyUI.
|
||
* Hard refresh your web browser (`Ctrl+F5` or `Cmd+Shift+R`).
|
||
|
||
---
|
||
|
||
|