Updated quantizing tool guide to compile with Visual Studio 2022 (#288)
This commit is contained in:
@@ -40,6 +40,8 @@ git apply ..\lcpp.patch
|
||||
|
||||
Compile the llama-quantize binary. This example uses cmake, on linux you can just use make.
|
||||
|
||||
### Visual Studio 2019, Linux, etc...
|
||||
|
||||
```
|
||||
mkdir build
|
||||
cmake -B build
|
||||
@@ -47,6 +49,31 @@ cmake --build build --config Debug -j10 --target llama-quantize
|
||||
cd ..
|
||||
```
|
||||
|
||||
### Visual Studio 2022
|
||||
|
||||
```
|
||||
mkdir build
|
||||
cmake -B build -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_FLAGS="-std=c++17"
|
||||
```
|
||||
|
||||
Edit the `llama.cpp\common\log.cpp` file, inserts two lines after the existing first line:
|
||||
|
||||
```
|
||||
#include "log.h"
|
||||
|
||||
#define _SILENCE_CXX23_CHRONO_DEPRECATION_WARNING
|
||||
#include <chrono>
|
||||
```
|
||||
|
||||
Then you can build the project:
|
||||
```
|
||||
cmake --build build --config Debug -j10 --target llama-quantize
|
||||
cd ..
|
||||
```
|
||||
|
||||
### Quantize your model
|
||||
|
||||
|
||||
Now you can use the newly build binary to quantize your model to the desired format:
|
||||
```
|
||||
llama.cpp\build\bin\Debug\llama-quantize.exe E:\models\unet\flux1-dev-BF16.gguf E:\models\unet\flux1-dev-Q4_K_S.gguf Q4_K_S
|
||||
|
||||
Reference in New Issue
Block a user