Spec 008-llamacpp-integration is complete; epic-finish bookkeeping
(archive) follows in a separate chore PR once this merges, matching
the llm-provider-abstraction epic's pattern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
LlamaCppProvider.list_models() caught every exception and returned [],
indistinguishable from "no models installed". Fixed _get_json to raise
on HTTP error status (matching _post_json's existing behavior — its
docstring already claimed this), and list_models() to distinguish
OSError (genuinely unreachable, degrades to []) from RuntimeError
(server responded with an error — surfaced with a router-mode hint).
Found by beacon-reviewer ahead of PR open.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Second LLMProvider (ADR-007), backed by llama-server's router mode. Only
one new ComfyUI node — LlamaCppClient, emitting the same LLM_CLIENT socket
Ollama Client does. ChatCompletion, LLMModelSelector, LLMLoadModel,
LLMUnloadModel work unmodified once wired to it — the actual proof the
provider abstraction generalizes, not just Ollama-shaped in practice.
API shape verified live against ggml-org/llama.cpp's tools/server/README.md
(research.md) rather than assumed: model identifier field is "id" (not
"name"), status is a nested {"value": "..."} object. list_models() needs
no normalization — llama.cpp's status vocabulary is exactly ModelStatus's
full set, unlike Ollama's narrower one.
38 new tests (test_llamacpp_provider.py, test_llamacpp.py), including a
dedicated US4 test proving no generic node branches on provider type —
the same call sequence succeeds against an Ollama-shaped or llama.cpp-
shaped fake provider.
README/docs updated for the new node this time, not left stale.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Full spec/plan/tasks/BDD scaffold for the llamacpp-integration epic, now
that its dependency (llm-provider-abstraction, PR #17) is satisfied.
Researched llama-server's router-mode API shape live (postdates training
data) rather than assuming it — two details that would have been wrong by
assumption: the model identifier field is "id" not "name" (differs from
Ollama's /api/tags), and "status" is a nested object ({"value": "..."}) not
a flat string.
Unlike the prerequisite epic, this decomposition genuinely holds as 4
independent user stories — LlamaCppClient is a new node, not a changed one,
so there's no shared "output type" migration forcing an atomic cutover.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
README.md and docs/index.md still named OllamaChatCompletion/OllamaModelSelector/
OllamaLoadModel/OllamaUnloadModel and the OLLAMA_CLIENT socket — stale after
the atomic cutover. Updated both (they're content-duplicates) and added the
old->new migration table FR-009 requires, which previously only existed as
a code comment (comfydv.ollama.MIGRATION_MAP), not surfaced to users.
Found via a dedicated docs-freshness pass, not caught by the beacon-reviewer
correctness review (which is scoped to code, not prose).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
chat_structured() silently dropped the options dict (temperature, seed,
num_predict, repeat_penalty, etc. — set via the OllamaOption* nodes) in
structured_output mode: OllamaProvider.chat_structured() only used it for
the cache key, and the shared comfydv._llm.chat.chat_structured() helper
had no options parameter at all. On main, the hand-rolled implementation
sent options verbatim in the request body; this migration silently lost it,
contradicting FR-008 (structured-output behavior must be unchanged).
Fixed by forwarding options as pydantic-ai's model_settings.extra_body,
matching the original payload shape exactly rather than lossily remapping
onto ModelSettings' own standardized field names (which don't cover
Ollama-native params like num_predict/repeat_penalty/top_k anyway).
Verified against the live local Ollama server: structured output with
temperature=0.0/seed=42 succeeds on first attempt.
Caught by an independent beacon-reviewer pass on the diff before opening a
PR — exactly what that review step is for.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Rewrote tests/test_ollama.py against a _FakeProvider double per
atomic-cutover-plan.md's D4/D5 test-layer split: node-contract/delegation
tests only, no aiohttp/_post_json mocking (that lives in
test_ollama_provider.py now). 98 unit tests, collapsed from the original
~125 largely mechanical monkeypatch-per-scenario tests where the retry/
validation coverage they duplicated already lives in
test_llm_chat_structured.py.
Fixed a real gap the rename surfaced along the way: comfy-manager-entry.json
(ComfyUI Manager registry listing) still had the old Ollama-specific
display names — updated it and its matching test expectation.
Full validation (T-CUT-10): 218 tests green (only the pre-existing,
unrelated Dockerfile-python-version test fails); ruff/ty clean (confirmed
via git stash comparison that the remaining ty diagnostics pre-date this
work); beacon doctor --strict shows only already-disclosed items.
Manual smoke test (T-CUT-12): Ollama was reachable in this environment, so
ran the real integration suite rather than a walkthrough. 6/8 passed
including the critical paths (real load/unload, real structured-output
retry-then-raise, real error handling). 2 failures traced via direct curl
to Ollama's /api/chat (bypassing this codebase) to a pre-existing,
ADR-006-documented unreliability in the specific test model, not a
regression.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
OllamaClient now emits an OllamaProvider via the generic LLM_CLIENT socket
(was OllamaClientType/OLLAMA_CLIENT). OllamaChatCompletion, OllamaModelSelector,
OllamaLoadModel, OllamaUnloadModel renamed to ChatCompletion, LLMModelSelector,
LLMLoadModel, LLMUnloadModel — all delegate to client.* (LLMProvider protocol)
instead of building Ollama URLs/payloads inline. Structured output now goes
through client.chat_structured() (pydantic-ai), not hand-rolled tool-calling.
ollama.py's own duplicate HTTP/cache infra (_post_json, _TTLLRUCache,
_MODEL_LIST_CACHE, _CHAT_RESPONSE_CACHE) is gone — single source of truth in
comfydv._llm.ollama_provider (plan D1); the combo-widget path
(_load_default_models, /dv/ollama/models) imports _fetch_models/_run_async
from there instead of duplicating them. _client_headers deleted (dead code
once nodes delegate to client.*).
Added MIGRATION_MAP (old->new node/socket names, FR-009). NODE_CLASS_MAPPINGS
keys changed to match (breaking rename, confirmed acceptable per ADR-007).
conftest.py's autouse cache-clearing fixture repointed to the single cache
source and ChatCompletion (plan D8).
tests/test_ollama.py itself is NOT yet updated (T-CUT-08, next commit) — it
will fail to collect until then; this is the expected atomic-cutover
intermediate state, not a regression to fix independently.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
list_models()/load_model()/unload_model()/chat()/chat_structured() —
behavior-preserving ports of the existing inline ollama.py logic, plus one
genuinely new capability: list_models() also queries /api/ps to report
real loaded/unloaded status (the old OllamaModelSelector only ever showed
names via /api/tags, never live status).
15 new tests in tests/test_ollama_provider.py, mocking at the module's own
_post_json/_get_json seam per atomic-cutover-plan.md's D4 test-layer split.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Struck-through historical tasks (- [ ] ~~...~~) still read as open
checkboxes to beacon's tooling — bullet status was picking one as "active"
instead of the real next step. Marked all superseded Phase 3/5/T014/T019-21
entries [-] (deferred, per the project's own convention), and added a real
Phase 8 with T-CUT-01..12 as trackable checkboxes matching
atomic-cutover-plan.md, so tooling and humans see the same source of truth.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Full line-by-line inventory of every reference in src/comfydv/ollama.py
and tests/test_ollama.py (1820 lines, read in full — not estimated)
affected by the OllamaClient/OllamaChatCompletion/OllamaModelSelector/
OllamaLoadModel/OllamaUnloadModel rename, plus the design decisions it
surfaced that "just rename it" was hiding:
- Single source of truth for HTTP/cache infra (comfydv._llm.ollama_provider)
- client == "<host string>" equality is removed, not preserved (OllamaProvider
isn't a str subclass)
- Bare-string client backward compatibility is removed (undocumented side
effect of the old str-subclass trick, never an intended feature)
- Test-layer split: node-contract/delegation tests stay in test_ollama.py
using a _FakeProvider double; Ollama-wire-protocol tests move to a new
test_ollama_provider.py — this is what actually resolves the 35 relocated
_post_json monkeypatches, rather than patching them 1:1 at a seam that no
longer exists
- Structured-output retry tests are not ported 1:1 — already covered by
tests/test_llm_chat_structured.py's existing 6 tests
Produces a 12-step sequenced task list (T-CUT-01..12) in
atomic-cutover-plan.md that supersedes tasks.md's original US1/US3
decomposition (struck through, kept for history). This is now the
authoritative plan for the next BUILD session to execute.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Provider-layer only — src/comfydv/_llm/chat.py doesn't touch ollama.py, so
it's safe to land ahead of the deferred atomic node cutover (issue #16).
Ports ADR-006's exact retry/validation contract: bounded retries (0-5,
clamped), RuntimeError naming model/attempt-count/truncated-last-response
on exhaustion. Agent's own internal retries disabled so the error contract
is comfydv's, not pydantic-ai's.
Required an API-discovery spike (Agent/OpenAIProvider/OpenAIChatModel
constructors and exception surface aren't reliably knowable from training
data for a fast-moving library) before tests could be written meaningfully
— tests and implementation were validated together rather than strictly
red-first; noted in tasks.md rather than presented as pure TDD.
Also fixes a pre-existing gap in test_packaging.py's requirements.txt/
pyproject.toml consistency check: it stripped `[extras]` on one side of the
comparison but not the other, which this change's pydantic-ai-slim[openai]
entry exposed as the first dependency with an extras marker.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Discovered mid-implementation: OllamaClient is a single shared producer for
every downstream Ollama node, so changing its output type per ADR-007
breaks all consumers simultaneously, and the class rename breaks
tests/test_ollama.py's 125 references atomically, not per-story. Confirmed
by independent product + engineering review (agent-trio deliberation,
aligned). Re-scoped the node-layer cutover as its own follow-up session,
tracked in issue #16. ADR-007's decision itself is unaffected — only
delivery sequencing changes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Adds pydantic-ai-slim[openai] per ADR-007, the Message/ModelStatus/
ModelInfo/LLMProvider protocol, and an OllamaProvider skeleton carrying
host/headers as instance state — ported _post_json/_fetch_models/_run_async/
_TTLLRUCache infra from comfydv.ollama unchanged. Method bodies land per
user story in specs/007-llm-provider-abstraction/tasks.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Adopt a shared LLMProvider protocol (list/load/unload/chat/structured-chat)
as the adapter pattern answer to GitHub issue #15 — Ollama and llama.cpp
share the chat/structured-output mechanism via pydantic-ai while model
lifecycle stays backend-specific per provider. Supersedes ADR-006, narrows
ADR-004's scope to non-chat REST calls.
Adds epic 007 (llm-provider-abstraction, prerequisite) and the
llamacpp-integration epic it unblocks, plus the full spec/plan/tasks/BDD
scaffold for the prerequisite epic.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132ojafeazQ3ephcBejEWFj
Adds a POST /dv/ollama/update_structured_outputs route and a matching JS
extension so editing structured_output/output_schema on
OllamaChatCompletion updates the node's output sockets immediately in the
UI — one per schema property — mirroring FormatString's existing live
dynamic-output pattern. No need to run the graph first to see the extracted
fields appear. Invalid/incomplete JSON while typing falls back to the base
3-output shape rather than erroring the request.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds structured_output/output_schema/max_retries inputs to
OllamaChatCompletion, fixing three real reliability problems: models
prepending commentary, wrapping responses in code fences, and occasionally
returning blank output. When enabled, requests go through Ollama's
OpenAI-compatible tool-calling endpoint (a single forced tool call matching
output_schema) instead of native /api/chat, and each response is validated
against a dynamically-built pydantic model — invalid/blank responses retry
with fresh network calls, exhausting into a clear RuntimeError rather than
silently passing bad data downstream. One additional ComfyUI output socket
is exposed per schema property, mirroring FormatString's existing
dynamic-output pattern. Fully backward compatible: structured_output
defaults off and non-structured behavior is untouched.
Ollama's native /api/chat "format" field (JSON-Schema-constrained decoding)
was tried first but proved unreliable against a real local model during
implementation — see ADR-006 for the full investigation and why
tool-calling was chosen over both native format and pydantic-ai (the latter
would reintroduce httpx/openai-sdk, reversing ADR-004).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
All owned specs shipped, so beacon epic finish moves it to archive/ and
flips status to Done. Clears the epic-spec-lifecycle and epic-gates
warnings beacon doctor --strict was raising.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the static project.version string with dynamic = ["version"],
sourced from the nearest git tag (hatch-vcs). Fixes docs.yml, which read
pyproject.toml's version field directly and would have broken once that
field stopped being a plain string; it now reads the resolved version via
importlib.metadata after `uv sync` installs the package.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
FormatString is now OUTPUT_NODE=True and returns {"ui": {"text": [...]},
"result": (...)} instead of a bare tuple, so the rendered output shows
directly on the node — no separate Show Text node needed to inspect it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds an OLLAMA_HEADERS socket type (Basic Auth, Bearer Token, Custom) that
composes onto OllamaClient, so requests to a proxied/authenticated Ollama
server carry the right Authorization header through model discovery,
load/unload, and chat.
Also adds a bounded local cache: a short-TTL LRU for model discovery
(avoids duplicate /api/tags round-trips from repeated node creation/refresh)
and an unbounded-TTL LRU for chat responses, keyed on the full request
signature — needed because OllamaChatCompletion is OUTPUT_NODE=True and so
is re-executed by ComfyUI on every queue run even when nothing changed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- README.md + docs/index.md: fix stale wiring instructions that referenced
model_name optional input (removed); now correctly shows wiring
OllamaLoadModel.model_name → OllamaChatCompletion.model (STRING)
- ChatCompletion description updated to mention inline response display
- Minimal chat workflow prose updated to reflect STRING model input + inline
display (no separate Text Display node needed)
- .github/workflows/docs.yml: auto-deploy docs via mike on push to main;
reads version from pyproject.toml, deploys as <version> + stable alias
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
- OllamaChatCompletion.model changed from COMBO to plain STRING so it can
receive a wired value from OllamaLoadModel without a separate model_name
socket (removes dual-input UX confusion and COMBO validation errors)
- Removed model_name optional input — model STRING is the single entry point
- _load_default_models() tries OLLAMA_HOST env var → host.docker.internal →
localhost so COMBO nodes pre-populate on server start inside Docker
- docker-compose.dev.yml: OLLAMA_HOST=http://host.docker.internal:11434 so
model dropdowns are populated at startup without a manual Refresh click
- JS: refreshModelWidget handles both COMBO and STRING widget types; keeps
user-typed/wired value when it is already a known model
- Tests: first_generative_model fixture dynamically selects the first
installed model instead of hardcoding embeddinggemma:latest (which is an
embedding-only model that rejects /api/generate with HTTP 400)
- Tests: test_multi_turn_receives_context passes think=False to prevent
Qwen3 thinking-mode models from returning empty content fields
- Screenshots retaken: ollama_chat.png, ollama_lifecycle.png, ollama_workflow.png
All 80 unit + 9 integration tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
OllamaChatCompletion now sets OUTPUT_NODE = True and returns
{"ui": {"text": [...]}, "result": (...)} so ComfyUI renders the
response text directly in the node body after each run.
On first turn, the full response is displayed.
On subsequent turns, the response is shown with a compact history
summary (last 6 messages, 100-char snippets) so multi-turn context
is visible at a glance without a separate display node.
_history_preview() helper added for the summary rendering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
5 failing unit tests:
- test_chat_is_output_node
- test_chat_returns_ui_result_dict
- test_chat_ui_contains_response_text
- test_chat_result_is_3_tuple
- test_wired_model_name_overrides_combo (updated to unpack from ["result"])
Integration tests updated to unpack from ["result"] for API consistency.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
OllamaChatCompletion grew taller with the new timeout_secs widget.
Nodes repositioned with generous horizontal gaps (60→380→720→1200)
and scale reduced to 0.72 so the full Load→Chat→Unload chain is
visible without overlap.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
9 new tests cover:
- _run_async: event-loop safety when called from a running loop
- _post_json: raises RuntimeError on 4xx/5xx; timeout kwarg forwarded
- LoadModel/UnloadModel: must use /api/generate, keep_alive must be int
- ChatCompletion: must expose timeout_secs widget and forward it
58 pre-existing unit tests remain green. Integration tests excluded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
OllamaUnloadModel now works as a pass-through node: wire
OllamaChatCompletion.response into the optional `passthrough` input and
OllamaChatCompletion.model_name into `model` — ComfyUI's data-flow graph
then guarantees Load → Chat → Unload execution order.
OllamaChatCompletion gains an optional `model_name` forceInput so
OllamaLoadModel.model_name can wire in (creating Load → Chat ordering) and
overrides the COMBO dropdown. It also returns `model_name` as a 3rd output
so the model can flow downstream to UnloadModel without duplication.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
Fresh Playwright sessions landed on ComfyUI's template gallery (no `workflow`
key in localStorage) instead of the node canvas. Pre-seeding an empty workflow
via Playwright storage_state fixes the capture. Also shows localhost:11434 in
the OllamaClient screenshot (user-facing default) rather than the Docker-internal
routing URL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
Mirrors the README.md rewrite into the MkDocs site landing page:
- Add Circuit Breaker section with screenshot (assets/circuit_breaker.png)
- Update Format String and Random Choice prose to match README
- Replace stale dependency list (rich/colorama/termcolor removed in spec 001)
- Add ComfyUI Manager install path
- Image paths use assets/ (MkDocs-relative) vs docs/assets/ in README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
Replace hand-crafted PowerPoint exports with Playwright-generated screenshots
taken from the actual running Docker test harness:
- scripts/take_screenshots.py: headless Chromium via Playwright; creates nodes
programmatically via LiteGraph/window.app, triggers updateNodeConfig() for
FormatString's dynamic socket update, then crops tightly around each node
- docs/assets/: four screenshots (fstring, jinja2, random, circuit_breaker)
replacing the old 1920×1080 PowerPoint exports (90%+ smaller, more accurate)
- Justfile: add `screenshots` recipe (starts harness if needed, waits healthy,
runs the script)
- README.md: rewrite to cover all three nodes including Circuit Breaker;
fix stale dependency list (was listing rich/colorama/termcolor which we
removed in spec 001); add ComfyUI Manager install path
- pyproject.toml + uv.lock: add playwright to [dev] dependencies
Run: just screenshots
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWysE6jYn4YjNorLBQ1cXb
