fix(workflows): disable thinking by default in LTX-2.3 pipeline (#34)
Every ChatCompletion node in ltx-i2v-pipeline.json now reads its options through a new OllamaOptionDisableThinking node (id 18, disable_thinking=True) appended to the end of the existing max-tokens/extra-body chain. Without it, a thinking-capable model routinely burns its whole max_tokens budget on chain-of-thought and never emits the closing JSON, so chat_structured fails validation against an empty string — confirmed live against saracen9/amoral-qwen3.5-9b. README's thinking-headroom section updated to match: thinking off by default, headroom advice now framed as what to do if you deliberately re-enable it via node 18. Claude-Session: https://claude.ai/code/session_01YArD9ZjBWKsAvazmS48amA Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
James Veitch
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
a613006f5a
commit
2724f535f5
+21
-16
@@ -37,23 +37,28 @@ round-trips.
|
||||
chat template is degenerate enough that it returns valid-shaped but
|
||||
garbled content regardless of structured-output mechanism (see
|
||||
[ADR-009](../project-management/ADRs/ADR-009-native-structured-output-mode.md)).
|
||||
- **If your model has "thinking" capability** (most current instruct
|
||||
models do), give it real headroom: nodes `16`/`17` set
|
||||
`max_tokens=8192`/`num_ctx=32768` for exactly this reason — a thinking
|
||||
model's chain-of-thought reasoning consumes real tokens before it ever
|
||||
emits the structured JSON, and Ollama's default context (4096, with
|
||||
- **Thinking is off by default.** Node `18` (`OllamaOptionDisableThinking`,
|
||||
`disable_thinking=True`) sits at the end of the options chain every
|
||||
`ChatCompletion` node reads from. Without it, a thinking-capable model
|
||||
routinely burns its whole `max_tokens` budget on chain-of-thought and
|
||||
never emits the closing JSON — `chat_structured` then fails validation
|
||||
against an empty string. Flip node `18`'s `disable_thinking` to `False`
|
||||
if you deliberately want a model to reason before answering; if you do,
|
||||
give it real headroom: nodes `16`/`17` set `max_tokens=8192`/`num_ctx=32768`
|
||||
for exactly that case, and Ollama's default context (4096, with
|
||||
`--context-shift` silently evicting old context rather than stopping)
|
||||
is nowhere near enough for these agents' long system prompts. Node `17`'s
|
||||
`num_ctx` reaches Ollama correctly because `OllamaProvider.chat_structured()`
|
||||
now calls Ollama's *native* `/api/chat` + `"format"` directly (ADR-009) —
|
||||
an earlier version of this fix tried priming context via a separate call
|
||||
before the real request and that didn't work, because Ollama's
|
||||
OpenAI-compatible endpoint silently reloads the model at its default
|
||||
context on every call, undoing any priming; the native endpoint doesn't
|
||||
have that problem and applies `options` and structured output atomically
|
||||
in one request. Every `ChatCompletion` node's `timeout_secs=600` for the
|
||||
same headroom reason; lower it if your hardware is faster than the
|
||||
machine this was tuned against.
|
||||
is nowhere near enough for these agents' long system prompts on top of
|
||||
reasoning tokens. Node `17`'s `num_ctx` reaches Ollama correctly because
|
||||
`OllamaProvider.chat_structured()` calls Ollama's *native* `/api/chat` +
|
||||
`"format"` directly (ADR-009) — an earlier version of this fix tried
|
||||
priming context via a separate call before the real request and that
|
||||
didn't work, because Ollama's OpenAI-compatible endpoint silently
|
||||
reloads the model at its default context on every call, undoing any
|
||||
priming; the native endpoint doesn't have that problem and applies
|
||||
`options` and structured output atomically in one request. Every
|
||||
`ChatCompletion` node's `timeout_secs=600` for the same headroom reason;
|
||||
lower it if your hardware is faster than the machine this was tuned
|
||||
against.
|
||||
- To use `LlamaCppClient` instead of `OllamaClient`, swap node `1`'s
|
||||
`class_type` and `host` — every `ChatCompletion` node keeps working
|
||||
unchanged, since both emit the same `LLM_CLIENT` type (ADR-007). Note
|
||||
|
||||
@@ -75,7 +75,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"contract_version\": {\"type\": \"string\"}, \"original_user_intent\": {\"type\": \"string\"}, \"duration_seconds\": {\"type\": \"number\"}, \"subject_references\": {\"type\": \"array\"}, \"required_changes\": {\"type\": \"array\"}, \"required_emotions\": {\"type\": \"array\"}, \"required_camera\": {\"type\": \"object\"}, \"required_audio\": {\"type\": \"object\"}, \"introduced_entities\": {\"type\": \"array\"}, \"preservation_requirements\": {\"type\": \"array\"}, \"prohibitions\": {\"type\": \"array\"}, \"allow_partial_completion\": {\"type\": \"boolean\"}, \"ambiguities\": {\"type\": \"array\"}}, \"required\": [\"contract_version\", \"original_user_intent\", \"subject_references\", \"required_changes\", \"required_emotions\", \"required_camera\", \"required_audio\", \"introduced_entities\", \"preservation_requirements\", \"prohibitions\", \"allow_partial_completion\", \"ambiguities\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -117,7 +117,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"manifest_version\": {\"type\": \"string\"}, \"audit_status\": {\"type\": \"string\"}, \"scene_summary\": {\"type\": \"object\"}, \"observed_entities\": {\"type\": \"array\"}, \"observed_relations\": {\"type\": \"array\"}, \"uncertain_elements\": {\"type\": \"array\"}, \"unverified_regions\": {\"type\": \"array\"}, \"continuity_locks\": {\"type\": \"object\"}, \"forbidden_unless_user_authorised\": {\"type\": \"array\"}}, \"required\": [\"manifest_version\", \"audit_status\", \"scene_summary\", \"observed_entities\", \"observed_relations\", \"uncertain_elements\", \"unverified_regions\", \"continuity_locks\", \"forbidden_unless_user_authorised\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -162,7 +162,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"verification\": {\"type\": \"object\"}, \"audited_manifest\": {\"type\": \"object\"}}, \"required\": [\"verification\", \"audited_manifest\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -221,7 +221,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"status\": {\"type\": \"string\"}, \"prompt\": {\"type\": \"string\"}, \"entity_resolution\": {\"type\": \"array\"}, \"intent_coverage\": {\"type\": \"array\"}, \"bridge_movements\": {\"type\": \"array\"}, \"referenced_entities\": {\"type\": \"array\"}, \"introduced_entities\": {\"type\": \"array\"}, \"preserved_entities\": {\"type\": \"array\"}, \"camera_delta\": {\"type\": \"object\"}, \"extra_changes\": {\"type\": \"array\"}, \"manifest_challenge\": {\"type\": \"object\"}, \"unsatisfiable_reason\": {\"type\": \"string\"}}, \"required\": [\"status\", \"entity_resolution\", \"intent_coverage\", \"bridge_movements\", \"referenced_entities\", \"introduced_entities\", \"preserved_entities\", \"camera_delta\", \"extra_changes\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -278,7 +278,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"decision\": {\"type\": \"string\"}, \"confidence\": {\"type\": \"number\"}, \"intent_audit\": {\"type\": \"array\"}, \"entity_audit\": {\"type\": \"array\"}, \"violations\": {\"type\": \"array\"}, \"missing_requirements\": {\"type\": \"array\"}, \"preserve_clauses\": {\"type\": \"array\"}, \"refinement_instruction\": {\"type\": \"string\"}}, \"required\": [\"decision\", \"confidence\", \"intent_audit\", \"entity_audit\", \"violations\", \"missing_requirements\", \"preserve_clauses\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -335,7 +335,7 @@
|
||||
"structured_output": true,
|
||||
"output_schema": "{\"type\": \"object\", \"properties\": {\"status\": {\"type\": \"string\"}, \"prompt\": {\"type\": \"string\"}, \"applied_fixes\": {\"type\": \"array\"}, \"intent_coverage\": {\"type\": \"array\"}, \"remaining_known_issues\": {\"type\": \"array\"}, \"unresolvable_reason\": {\"type\": \"string\"}}, \"required\": [\"status\", \"applied_fixes\", \"intent_coverage\", \"remaining_known_issues\"]}",
|
||||
"options": [
|
||||
"17",
|
||||
"18",
|
||||
0
|
||||
],
|
||||
"timeout_secs": 1200,
|
||||
@@ -364,5 +364,18 @@
|
||||
4
|
||||
]
|
||||
}
|
||||
},
|
||||
"18": {
|
||||
"class_type": "OllamaOptionDisableThinking",
|
||||
"_meta": {
|
||||
"title": "Disable Thinking (default \u2014 avoids empty/truncated structured output on thinking models)"
|
||||
},
|
||||
"inputs": {
|
||||
"disable_thinking": true,
|
||||
"options": [
|
||||
"17",
|
||||
0
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user