82 lines
2.3 KiB
Markdown
82 lines
2.3 KiB
Markdown
# `llama-cpp-python` CUDA Installation for ComfyUI Portable
|
||
|
||
This guide helps you install `llama-cpp-python` with GPU (CUDA) support for ComfyUI's Windows portable version.
|
||
|
||
---
|
||
|
||
### **1. Open ComfyUI's Command Prompt**
|
||
|
||
* Go to `C:\ComfyUI_windows_portable\python_embeded`
|
||
* In the address bar, type `cmd` and press Enter.
|
||
|
||
---
|
||
|
||
### **2. Install Build Tools (If you haven't already)**
|
||
|
||
* **Install Visual Studio Build Tools:**
|
||
* Download from [https://visualstudio.microsoft.com/downloads/](https://visualstudio.microsoft.com/downloads/) (under "Tools for Visual Studio").
|
||
* Run installer, select **"Desktop development with C++"** workload.
|
||
* **Install NVIDIA CUDA Toolkit:**
|
||
* Download your CUDA version (e.g., 12.6) from [https://developer.nvidia.com/cuda-toolkit-archive](https://developer.nvidia.com/cuda-toolkit-archive).
|
||
* Run installer, ensure **"Visual Studio Integration"** is selected.
|
||
|
||
## **Automatic Installation Script (Recommended)**
|
||
|
||
* Make sure the installation script is saved here:
|
||
`.\ComfyUI\custom_nodes\ComfyUI-MiniCPM\install_llama_official.py`
|
||
|
||
* In the command prompt (opened in step 1), run the script with:
|
||
|
||
```bash
|
||
.\python_embeded\python.exe install_llama_official.py
|
||
```
|
||
|
||
The script will:
|
||
|
||
* Upgrade pip
|
||
* Clear pip cache
|
||
* Install minimal build dependencies (scikit-build-core, cmake)
|
||
* Detect GPU and build llama-cpp-python with CUDA support if available
|
||
|
||
🕒 This process may take 5–20+ minutes depending on your system.
|
||
|
||
### ** Restart ComfyUI**
|
||
|
||
* Close ComfyUI.
|
||
* Restart ComfyUI.
|
||
* Hard refresh your browser with Ctrl+F5 (Windows) or Cmd+Shift+R (macOS).
|
||
|
||
|
||
## **Manuelly Installation**
|
||
---
|
||
|
||
### **1. Prepare Python Environment**
|
||
|
||
* In the command prompt, run:
|
||
```bash
|
||
.\python.exe -m pip install --upgrade pip
|
||
.\python.exe -m pip cache purge
|
||
.\python.exe -m pip install scikit-build-core cmake
|
||
```
|
||
|
||
---
|
||
|
||
### **2. Compile and Install `llama-cpp-python`**
|
||
|
||
* In the command prompt, run:
|
||
```bash
|
||
set CMAKE_ARGS="-DGGML_CUDA=on" && .\python.exe -m pip install llama-cpp-python --no-cache-dir && set CMAKE_ARGS=
|
||
```
|
||
* **Wait patiently.** This takes time (5-20+ minutes).
|
||
|
||
---
|
||
|
||
### **3. Restart ComfyUI**
|
||
|
||
* Close ComfyUI.
|
||
* Restart ComfyUI.
|
||
* Hard refresh your web browser (`Ctrl+F5` or `Cmd+Shift+R`).
|
||
|
||
---
|
||
|