diff --git a/tools/README.md b/tools/README.md index fb962c3..38625f8 100644 --- a/tools/README.md +++ b/tools/README.md @@ -1,3 +1,12 @@ +## STEP 1 (Patch files with Unix (LF) line endings + +Solution to fix lines endings of the patch files from Windows (CRLF) to Unix (LF) + +``` +python fix_lines_ending.py +``` +## STEP 2 (Clone llama.cpp version of gguf-py) + This needs the llama.cpp version of gguf-py to work at the moment, not the pip one as that one does not have the python quantization code yet. ``` @@ -5,23 +14,31 @@ git clone https://github.com/ggerganov/llama.cpp pip install llama.cpp/gguf-py ``` +## STEP 3 (Convert to FP16 or BF16) To convert your initial source model to FP16 (or BF16), run the following command: ``` python convert.py --src E:\models\unet\flux1-dev.safetensors ``` +## STEP 4 (Patch llama.cpp) - -To quantize the model, first apply the provided patch to the llama.cpp repo you've just cloned. If you get a "corrupt patch" error, you may have to [change the line endings in the patch file](https://github.com/city96/ComfyUI-GGUF/issues/90#issuecomment-2323011648). +- To quantize the model, first apply the provided patch to the llama.cpp repo you've just cloned. ``` cd llama.cpp git checkout tags/b3600 git apply ..\lcpp.patch ``` -If you wish to quantize **SD3** or **AuraFlow** models, you should use the patch named `lcpp_sd3.patch`, which has slightly modified logic for quantizing. For this you'll want to target `tags/b3962` instead. +- To quantize **SD3** or **AuraFlow** models, you should use the patch `lcpp_sd3.patch` and target to `tags/b3962` instead. +``` +cd llama.cpp +git checkout tags/b3962 +git apply ..\lcpp_sd3.patch +``` +## STEP 5 (Compile llama-quantize binary) + Then, compile the llama-quantize binary. This example uses cmake, on linux you can just use make. ``` mkdir build @@ -32,7 +49,7 @@ cd .. cd .. ``` - +## STEP 6 (Quantization) Now you can use the newly build binary to quantize your model to the desired format: ``` llama.cpp\build\bin\Debug\llama-quantize.exe E:\models\unet\flux1-dev-BF16.gguf E:\models\unet\flux1-dev-Q4_K_S.gguf Q4_K_S diff --git a/tools/fix_lines_ending.py b/tools/fix_lines_ending.py new file mode 100644 index 0000000..346e350 --- /dev/null +++ b/tools/fix_lines_ending.py @@ -0,0 +1,31 @@ +import os + +files = ["lcpp.patch", "lcpp_sd3.patch"] + +def has_unix_line_endings(file_path): + try: + with open(file_path, 'rb') as file: + content = file.read() + return b'\r\n' not in content + except Exception as e: + print(f"Error checking '{file_path}': {e}") + return False + +def convert_to_linux_format(file_path): + try: + with open(file_path, 'rb') as file: + content = file.read().replace(b'\r\n', b'\n') + with open(file_path, 'wb') as file: + file.write(content) + print(f"'{file_path}' converted to Linux line endings (LF).") + except Exception as e: + print(f"Error processing '{file_path}': {e}") + +for file in files: + if os.path.exists(file): + if has_unix_line_endings(file): + print(f"'{file}' already has Unix line endings (LF). No conversion needed.") + else: + convert_to_linux_format(file) + else: + print(f"File '{file}' does not exist.")