nodes/godmt-comfyui-openai-llm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
main
godmt-comfyui-openai-llm/docs/thinking-controls.md
T
godmt 270d57e5f9 thinking update for local llm
2026-08-26 08:43:02 +09:00

4.6 KiB
Raw Permalink Blame History

Thinking Controls

This note captures reasoning/thinking request fields for the OpenAI-compatible LLM node.

There is no single universal OpenAI-compatible thinking switch. Different servers expose different request fields, and some local models implement thinking in their chat template rather than as a standardized API parameter.

The node stays within its scope: it sends fields on the configured OpenAI-compatible request body. It does not switch to provider-native endpoints such as LM Studio /api/v1/chat or Ollama /api/chat.

Automatic Behavior

In normal workflows, set thinking only. thinking_api_style is an advanced, normally collapsed control and defaults to auto. The old none value is treated as auto for compatibility with saved workflows. thinking: default still omits all thinking controls.

Endpoint/provider Automatic request field
OpenAI-compatible Chat Completions, including normal LM Studio/Ollama /v1 use reasoning_effort
Responses API reasoning.effort
OpenRouter URL reasoning.effort or reasoning.enabled
DashScope URL enable_thinking

This is endpoint-based rather than model-family-based. Gemma and Qwen do not need separate user-facing modes when their server supports the common field.

Advanced Overrides

Only choose an explicit thinking_api_style when the server rejects or ignores the automatic field. The most important exception is chat_template_kwargs_enable_thinking, for local Jinja templates that directly read chat_template_kwargs.enable_thinking. ollama_think is retained for an OpenAI-compatible endpoint that explicitly accepts think.

Qwen3/SlopPrompt Symptom

OpenAICompatibleLLM returns an empty text output while the expected answer is present in reasoning_text.

In llama.cpp chat completions responses this usually means:

  • message.reasoning_content contains the generated text.
  • message.content is empty.

The node maps content to text and reasoning_content to reasoning_text, so this split is visible in ComfyUI.

Cause

Some Qwen3-style Jinja chat templates prefill a thinking block when thinking is enabled. If the model finishes inside that block without emitting a closing </think> and final answer, a compatible server may classify the generated text as reasoning.

For SlopPrompt-like models, llama.cpp's web UI can avoid this by disabling thinking with enable_thinking=false.

Node Fix

If automatic mode is ignored by this particular server/template, use:

thinking_api_style: chat_template_kwargs_enable_thinking
thinking: off

This sends the following OpenAI-compatible request body fragment:

{"chat_template_kwargs": {"enable_thinking": false}}

Use this for compatible servers and models whose template reads chat_template_kwargs.enable_thinking.

qwen_chat_template_kwargs and llama_cpp_chat_template_kwargs are kept as legacy aliases for existing workflows, but new workflows should use chat_template_kwargs_enable_thinking.

Gemma Notes

Gemma 4 thinking is also template-driven in many runtimes, but it is not exactly the same behavior as Qwen3. Google documents Gemma 4 thinking through the chat template's enable_thinking flag in Transformers, and some local runtimes expose this through prompt-template editing, custom fields, or provider-specific reasoning controls.

For Gemma on LM Studio or Ollama /v1/chat/completions, use auto; it sends reasoning_effort. For a local OpenAI-compatible server template that ignores that field but explicitly reads enable_thinking, override with chat_template_kwargs_enable_thinking.

Server-Level Alternative

The same default can be set when starting llama-server:

.\llama-server.exe -m "D:\models\SlopPrompt\SlopPrompt-v5-Q8_0.gguf" -ngl 99 --jinja -c 8192 --host 127.0.0.1 --port 8080 --chat-template-kwargs "{\"enable_thinking\": false}"

Use the node setting when you want to control this per workflow/request instead of at server startup.

References

Reference in New Issue View Git Blame Copy Permalink