From e2eef3bbb8db5177ccb95683670fb91f3f02afc0 Mon Sep 17 00:00:00 2001 From: Nojahhh Date: Sun, 20 Oct 2024 23:42:07 +0200 Subject: [PATCH] clarified quant option when using GPTQ models of vision model --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d40db62..5d00ab5 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ The `GLM4ModelLoader` class is responsible for loading GLM-4 models. It supports `THUDM/glm-4v-9b` requires `bf16` and is set to run in 4-bit by default based on it's size. `alexwww94/glm-4v-9b-gptq-4bit` requires `bf16` and is set to run in 4-bit by default. `alexwww94/glm-4v-9b-gptq-3bit` requires `bf16` and is set to run in 3-bit by default. -- **quantization**: Set the number of bits for quantization (`4`, `8`, `16`). Default value of `4`. +- **quantization**: Set the number of bits for quantization (`4`, `8`, `16`). Default value of `4`. (This option is bypassed when using the GPTQ-models). #### Output