diff --git a/README.md b/README.md index 71c0bda..d376d8c 100644 --- a/README.md +++ b/README.md @@ -235,6 +235,8 @@ The node accepts a prompt and optional system prompt, an `IMAGE` batch, one nati Enter either a curated profile name from [`llm_profiles.json`](llm_profiles.json) or a Hugging Face `owner/model` ID directly (use `owner/model@revision` to pin a branch, tag, or commit). Each target inspects the model's metadata and compatibility before downloading weights, pins the exact commit, and reuses the completed snapshot on later runs. Unknown or target-incompatible architectures fail inspection before any weight download. Public models need no token; for gated models, set `HF_TOKEN` in ComfyUI's environment (local), in the Modal secret collection (Modal), or in the host's Docker env file (SSH). +**GGUF exception:** do not enter the raw Hugging Face repository ID for a GGUF model. Raw `owner/model` values use the Transformers-compatible resolver, which requires `config.json` and safetensors; GGUF repositories normally provide neither. Select a curated GGUF profile instead so Modal-Sync knows the exact `.gguf` weight, optional multimodal projector, tokenizer source, and `llama.cpp` backend. For example, use `huihui-qwen3.8-27b-abliterated-q2-k-gguf`, not `huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF`. + Apple-local inference requires macOS on Apple Silicon and the optional pinned runtime in the ComfyUI Python environment: ```bash diff --git a/docs/LEARNINGS.md b/docs/LEARNINGS.md index 271b492..1e6c4ae 100644 --- a/docs/LEARNINGS.md +++ b/docs/LEARNINGS.md @@ -489,6 +489,7 @@ - Nodes 2.0 node bodies can cover legacy canvas foreground drawings, so execution-location UI needs a DOM-layer equivalent rather than relying only on `onDrawForeground`. Attach the exact runtime identity to streamed status events as well as numeric progress; otherwise loader and other non-numeric phases know the active node but cannot display its Modal task ID. ## 2026-08-23: Vast instances need a direct-container backend +- A raw Hugging Face repository ID is valid only for the Transformers-compatible model resolver, which requires `config.json` and safetensors. GGUF repositories must be selected through a curated profile that pins the exact weight, optional projector, tokenizer source, and `llama.cpp` backend; otherwise prompt validation fails before provider placement. - Vast instances are already containers and do not support Docker-in-Docker, so they cannot be modeled as ordinary `SshDockerController` hosts. Reuse the provider-neutral planner, framing protocol, worker kernel, serialization, progress, cancellation, and artifact paths, but add direct SSH runtime and filesystem storage adapters. - Treat the disconnected Vast configuration node as a local v3 output sink. This preserves it in the queued prompt without wiring it into user data flow, and it must remain in the local-only class set during automatic placement. - Revalidate every offer response locally even when the marketplace query contains identical hard filters. Rank compatible offers deterministically by total hourly price, then reliability, DLPerf, download rate, and offer ID.