Fix thinking bleed: decode with skip_special_tokens=False to preserve delimiters

Qwen3 encodes <think> and </think> as special token IDs (151648/151649).
With skip_special_tokens=True they are stripped silently before the regex
runs, leaving raw reasoning content concatenated with the response and no
way to separate them. The thinking content then bleeds into the repaired
prompt.

Fix: always decode with skip_special_tokens=False so the thinking delimiters
are present in the string. Strip complete <think>...</think> blocks first,
then truncated blocks (no closing tag, model ran out of budget), then clean
remaining special tokens like <|im_end|> with a separate pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
digital-garbage
2026-05-16 15:34:40 +03:00
co-authored by Claude Sonnet 4.6
parent 6e04f96a5e
commit 8a81f640d2
+11 -4
View File
@@ -11248,10 +11248,17 @@ class _FunPackAdvisorLLMWrapper:
output_ids = generated["output_ids"]
prompt_length = generated.get("prompt_length", 0)
new_tokens = output_ids[0][prompt_length:]
text = self._tokenizer.decode(new_tokens, skip_special_tokens=skip_special_tokens)
# Strip chain-of-thought thinking blocks emitted by models like Qwen3.
text = re.sub(r"<think>[\s\S]*?</think>", "", text, flags=re.IGNORECASE).strip()
return text
# Decode with special tokens preserved so thinking delimiters are visible
# for stripping. Models like Qwen3 encode <think>/</ think> as special token
# IDs - with skip_special_tokens=True they vanish silently and the raw
# reasoning content bleeds into the output as if it were the response.
text = self._tokenizer.decode(new_tokens, skip_special_tokens=False)
# Strip complete thinking blocks first, then any truncated one (no closing tag).
text = re.sub(r"<think>[\s\S]*?</think>", "", text, flags=re.IGNORECASE)
text = re.sub(r"<think>[\s\S]*", "", text, flags=re.IGNORECASE)
# Strip remaining special tokens: <|im_end|>, <|endoftext|>, etc.
text = re.sub(r"<\|[^|]*\|>", "", text)
return text.strip()
_FUNPACK_ADVISOR_LLM_CACHE = {}