- Add _NO_THINK suffix to all system prompts instructing model not to reason
- Add robust _strip_thinking() function that handles:
- Tagged <think>...</think> blocks (complete and incomplete)
- Untagged reasoning text (Okay let me..., Wait..., The input is..., etc.)
- Mixed output where prompt is on first line then reasoning follows
- Import re module for regex-based cleaning
- Tested with Qwen3-4B-Q4_K_M in basic/detailed/creative modes: ALL CLEAN
Major improvements:
- Automatic GPU to CPU fallback with VRAM management
- Free PyTorch CUDA cache before each load attempt
- CPU-pure mode: op_offload=False, flash_attn=disabled, offload_kqv=False
(prevents llama.cpp from allocating ~606 MiB CUDA compute buffers)
- Progressive context reduction (4096 -> 2048 -> 512)
- Vision-Language model detection and filtering (Qwen3VL, LLaVA, etc.)
- Proactive CUDA DLL registration for Windows
- Always-verbose mode for native llama.cpp diagnostics
Ezequiel Tojo