Fix thinking bleed: decode with skip_special_tokens=False to preserve delimiters
Qwen3 encodes <think> and </think> as special token IDs (151648/151649). With skip_special_tokens=True they are stripped silently before the regex runs, leaving raw reasoning content concatenated with the response and no way to separate them. The thinking content then bleeds into the repaired prompt. Fix: always decode with skip_special_tokens=False so the thinking delimiters are present in the string. Strip complete <think>...</think> blocks first, then truncated blocks (no closing tag, model ran out of budget), then clean remaining special tokens like <|im_end|> with a separate pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 4.6
parent
6e04f96a5e
commit
8a81f640d2
+11
-4
@@ -11248,10 +11248,17 @@ class _FunPackAdvisorLLMWrapper:
|
||||
output_ids = generated["output_ids"]
|
||||
prompt_length = generated.get("prompt_length", 0)
|
||||
new_tokens = output_ids[0][prompt_length:]
|
||||
text = self._tokenizer.decode(new_tokens, skip_special_tokens=skip_special_tokens)
|
||||
# Strip chain-of-thought thinking blocks emitted by models like Qwen3.
|
||||
text = re.sub(r"<think>[\s\S]*?</think>", "", text, flags=re.IGNORECASE).strip()
|
||||
return text
|
||||
# Decode with special tokens preserved so thinking delimiters are visible
|
||||
# for stripping. Models like Qwen3 encode <think>/</ think> as special token
|
||||
# IDs - with skip_special_tokens=True they vanish silently and the raw
|
||||
# reasoning content bleeds into the output as if it were the response.
|
||||
text = self._tokenizer.decode(new_tokens, skip_special_tokens=False)
|
||||
# Strip complete thinking blocks first, then any truncated one (no closing tag).
|
||||
text = re.sub(r"<think>[\s\S]*?</think>", "", text, flags=re.IGNORECASE)
|
||||
text = re.sub(r"<think>[\s\S]*", "", text, flags=re.IGNORECASE)
|
||||
# Strip remaining special tokens: <|im_end|>, <|endoftext|>, etc.
|
||||
text = re.sub(r"<\|[^|]*\|>", "", text)
|
||||
return text.strip()
|
||||
|
||||
|
||||
_FUNPACK_ADVISOR_LLM_CACHE = {}
|
||||
|
||||
Reference in New Issue
Block a user