For LM Studio, Ollama, and custom OpenAI-compatible endpoints,
automatically append /v1 to the URL if missing. This prevents
the common "Unexpected endpoint or method" error when users
enter URLs like "http://127.0.0.1:1234" instead of the required
"http://127.0.0.1:1234/v1".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Models with "/" in their name (e.g., meta-llama/Llama-Vision-Free) are
now grouped by prefix, making the dropdown easier to navigate. Models
without "/" appear first, followed by grouped models sorted by prefix.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added validation for OpenAI-style API responses in _call_llm and
_analyze_component methods. LM Studio sometimes returns empty/null
choices array which caused 'NoneType is not subscriptable' crash.
Now provides clear error messages:
- "LLM returned None response - check if model is loaded"
- "LLM returned invalid response format"
- "LLM returned empty choices array"
- "LLM returned empty message content - model may have failed to generate"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Local vision models (QwenVL, Phi-3.5, etc.) output chain-of-thought
mixed with results when reasoning is enabled, breaking JSON parsing
for agentic mode. Single-shot prompts work much better.
Changes:
- Removed enable_reasoning toggle from SID_LLM_Local UI
- Hardcoded reasoning_enabled = False
- Local models now always use single-shot pipeline
Cloud APIs (Anthropic, OpenAI, Gemini) still support reasoning mode.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Updated framing component (components.toml) with comprehensive analysis:
- Body visibility tracking (head to feet, top to bottom)
- Frame cutoff detection (where body is cropped)
- Subject distance/scale (fills frame / medium / far)
- Rule of thirds positioning (horizontal + vertical)
- Enhanced JSON output schema for agentic mode
- Updated single-shot prompts (prompts.toml) for all tiers:
- Advanced tier: detailed/extreme with FRAMING & VISIBILITY section
- Standard tier: framing info in detailed/extreme modes
- Basic tier: updated detailed/extreme with body visibility and positioning
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Updated Detailed and Extreme mode prompts (all tiers) to include:
Body Positioning (each limb independently):
- Left/right shoulder position and angle
- Left/right arm position, bend, hand placement
- Left/right leg position if visible
- Weight distribution
Light & Shadow Analysis:
- Face: highlight areas (forehead, cheekbones), shadow areas (under eyes, jawline)
- Chest/torso: where light and shadow fall
- Arms: light and shadow placement
- Overall direction, key/fill ratio, rim lighting, shadow color temp
This ensures single-shot mode captures the same detail as agentic mode
with the new body_positioning and light_shadow_body components.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: Global CUDA settings were interfering with ComfyUI's diffusion:
1. cudnn.benchmark=True causes re-benchmarking when input sizes change
between local model inference and KSampler, adding significant overhead
2. TF32 settings may affect precision of subsequent operations
These optimizations were designed for single-use scenarios, not shared
environments like ComfyUI where multiple nodes use different operations.
Solution: Remove all global CUDA modifications. ComfyUI manages its own
CUDA state optimally. Our cleanup method still ensures proper state
restoration after inference.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New components for detailed prompt detection:
1. body_positioning - Detailed limb analysis:
- Left/right shoulder position, angle, visibility
- Left/right arm position, bend angle, hand placement
- Left/right leg position, bend, weight distribution
- All limbs analyzed independently
2. light_shadow_body - Light and shadow on body parts:
- Face: highlights (forehead, cheekbones), shadows (under eyes, jawline)
- Chest/torso: highlight and shadow areas
- Arms/hands: light and shadow placement
- Overall: main direction, key/fill ratio, rim lighting, shadow color temp
Added to "detailed" and "extreme" analysis modes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: setup_cuda_optimizations() was calling torch.set_grad_enabled(False)
globally at module load, which disabled gradient computation for ALL nodes in
ComfyUI, breaking VAE decode and other nodes that need gradients.
Changes:
- Remove the global torch.set_grad_enabled(False) call
- Enhanced _cleanup_after_generation() to:
- Restore gradient computation with torch.set_grad_enabled(True)
- Force garbage collection after CUDA cleanup
- Ensure clean GPU state for subsequent nodes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add _cleanup_after_generation() method that:
- Calls torch.cuda.synchronize() to ensure all CUDA operations complete
- Calls torch.cuda.empty_cache() to free GPU memory
This prevents ComfyUI's subsequent nodes (like VAE decode) from getting
stuck due to pending CUDA operations or fragmented GPU memory.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add prompt_length dropdown (Short/Medium/Long/Free) to control output length
- Default to "Free" (no constraint) for maximum flexibility
- Updated prompts.toml with Z-Image best practices:
- Structured format: composition → subject → clothing → lighting
- Geometric face angles (front view, 45°, profile)
- Specific lighting keywords (soft diffused daylight, rim lighting)
- Cleanup constraints at end (no text, no watermark)
- Added get_length_constraint() to config_loader.py
- Length constraints applied to both single-shot and agentic pipelines
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Moondream2: Disable bitsandbytes quantization (vision encoder has dtype
mismatch with Half/Float in layer_norm). Always use BF16/FP16 instead.
- Florence-2: Use num_beams=1 for greedy decoding (beam search has cache
compatibility issues with newer transformers causing NoneType errors)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Phi-3.5 Vision marked as fastest local model
- Updated recommendation to note QwenVL for quality, Phi for speed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
QwenVL currently produces the best quality prompts among local models.
Updated table and added recommendation note.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Florence-2: Cast input tensors to model dtype (fixes BFloat16 mismatch)
- Moondream2: Use HF repo with force_download on cache corruption retry
- DynamicCache: Clear sys.modules cache to prevent stale imports
- Cache clearing: Also remove transformers_modules Python module entries
Fixes:
- RuntimeError: Input type (float) and bias type (struct c10::BFloat16)
- FileNotFoundError: .../transformers_modules/Moondream2/layers.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The compatibility shim now handles both old and new transformers versions:
- Try get_seq_length() method first (newer transformers)
- Fall back to key_cache attribute access (older transformers)
- Return 0 if neither method is available
This fixes AttributeError: 'DynamicCache' object has no attribute 'key_cache'
when using Phi-3.5-Vision with newer transformers versions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add documentation that local models perform better with reasoning disabled
(single-shot mode) since they struggle with structured JSON output required
for agentic analysis.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Local Thinking models work better with single-shot mode rather than
agentic/reasoning mode which requires JSON output parsing. API providers
keep reasoning enabled by default as they handle structured output better.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add raw text fallback in _call_reasoning_llm when JSON parsing fails
- Add _clean_thinking_text method to extract useful content from thinking models
- Handle _raw_fallback_text in _execute_agentic to prevent empty prompts
- Include component_results in metadata JSON to show LLM response data
This fixes the empty prompt issue for local Thinking models (like Qwen3-VL-4B-Thinking)
in agentic/reasoning mode where the model outputs reasoning text instead of JSON.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Better handling of <think>...</think> tag extraction
- If nothing after </think>, extract from inside the tags
- Always return something (cleaned text or original) rather than empty
- Added more JSON extraction patterns for agentic mode output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add _clean_thinking_output() method to remove reasoning from output
- Handle <think>...</think> tags from QwenVL Thinking models
- Detect and skip plain-text reasoning patterns (Let me, We are given, etc.)
- Extract actual answer from JSON blocks or after common delimiters
- Fall back to last substantial paragraph when reasoning pattern detected
Fixes issue where Qwen3-VL Thinking models would output their entire
reasoning chain instead of just the final prompt.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove [MODIFICATION: ...] fallback tag that was appearing in output
- Add string-based enhancement for local vision models since they can't
do text-only LLM calls reliably
- Add _apply_string_enhancement() to intelligently parse and apply user
instructions like "convert the dress into yellow colour"
- Handle common patterns: "make X", "change to X", "convert X into Y"
- Fall back to calling enhancement on agentic pipeline when user_focus
not extracted from JSON
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Best reasoning model in the Qwen3-VL lineup:
- 8B parameters with thinking/reasoning capability
- VRAM: 4.5GB (4-bit), 7GB (8-bit), 12GB (FP16)
- 4K max output tokens
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Florence-2: Force eager attention to avoid _supports_sdpa error
(Florence-2's custom code doesn't define this attribute)
- Moondream2: Add cache corruption detection and auto-cleanup
(clears transformers_modules cache and retries on FileNotFoundError)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Apply DynamicCache compatibility patch to:
- Florence-2
- Moondream2
- SmolVLM
- Phi-3.5-Vision (already had it)
Qwen works fine without the patch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Phi-3.5-Vision model calls get_usable_length(kv_seq_len, layer_idx)
with two arguments, not just one.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add get_usable_length method to DynamicCache compatibility shim for
newer transformers versions that removed this method.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add _enhance_prompt_with_instructions() method that makes a second
LLM call to intelligently apply user modifications to generated prompts
- Fix DynamicCache compatibility for newer transformers versions
(seen_tokens and get_max_length attributes)
- Fix output token extraction to only decode new tokens, preventing
system prompt from appearing in output
- Improve user guidance handling with MODIFICATION directive in
system prompts for better instruction following
- Only call enhancement LLM when user_guidance is provided
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add subject-type-aware component filtering for LANDSCAPE/VEHICLE/ANIMAL/PRODUCT
- Add new components: landscape_terrain, landscape_weather, landscape_scenery,
vehicle_details, animal_details, product_details
- Update framing component to be subject-neutral (removed human-specific pose)
- Add SUBJECT_COMPONENTS mapping for appropriate analysis per subject type
- Add metadata and debug string outputs to prompt generator node
- Enhance user guidance to be contextually integrated with image analysis
instead of simple [FOCUS:] prefix
- Remove redundant user guidance prepending in single-shot mode
Fixes landscape images incorrectly getting human-centric descriptions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The flash_attn module could be imported but was missing package metadata,
causing transformers to fail when checking the version. Now we properly
check importlib.metadata.version() before enabling flash_attention_2,
falling back to SDPA if metadata is missing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add comprehensive body_figure component to analyze:
- Nudity/exposure level (clothed to nude)
- Breast exposure details (covered/cleavage/sideboob/underboob)
- Cup size estimation (AA/A through G+)
- Body type, bust shape, cleavage visibility and shadows
- Waist definition, hip curves, buttocks shape
- Thigh fullness and body contour shadows
- Update DETAIL_MODES to include body_figure in Standard/Detailed/Extreme
- Update component order array for prompt assembly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add sample workflows for Ollama, Groq, OpenRouter, and local QwenVL
- Add vision model warning for Ollama/LM Studio non-vision models (warn but allow)
- Improve basic tier prompts with examples for local models
- Remove numbered lists in favor of flowing instructions
- Add more cleanup patterns for local model output prefixes
- Fix issues #4 (sample workflows) and #5 (local model prompts)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add attention_mode selection (auto/flash_attention_2/sdpa/eager)
- Add torch.compile support for CUDA acceleration (~20-30% speedup)
- Add repetition_penalty and top_p parameters for better output control
- Implement response validation with retry logic for short responses
- Add adaptive resolution based on Qwen model version (Qwen3 vs Qwen2.5)
- Add quality metrics tracking (word count, tokens/sec)
- Add provider-specific cleanup patterns (Groq, Together, Fireworks, etc.)
- Update providers.toml with local model configurations and VRAM guidelines
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements externalized configuration using TOML files for:
- Provider tiers (advanced/standard/basic) and stop strings
- Tier-specific system/user prompts for local vs cloud models
- Component prompts for agentic analysis by tier
- Output filter patterns for example text removal
Local models (Ollama, LM Studio) now get simplified prompts
without JSON output or complex numbered instructions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete SID_GGUF_LLM node and related files
- Remove llm_providers/sid_gguf_llm.py
- Remove prompt_templates/local_gguf_template.py
- Remove sample_workflows/sample_workflow_gguf.json
- Remove GGUF provider from generator nodes
- Simplify template_factory.py to use Claude template for all providers
- Update pyproject.toml keywords
- QwenVL is now the recommended local LLM provider
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added class-level image cache (hash -> processed tensors)
- Cache pixel_values and image_grid_thw between calls with same image
- LRU-style cache with max 3 images (for component mode)
- Significantly speeds up component mode (8+ calls per image)
- Already had model caching and KV cache enabled
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Updated all Anthropic API calls to use cache_control with ephemeral type
- Cache reads are 0.1x the cost of input tokens (90% savings)
- Latency reduction up to 85% on cached prompts
- Cache lasts 5 minutes, refreshed on each use
- Applied to: Basic generator, Advanced V2 generator, iterative_improve.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added enable_reasoning toggle to SID_QwenVL_LLM, SID_Grok_LLM, SID_GGUF_LLM
- Removed prompt_mode dropdown from Advanced V2 generator
- Generator now auto-detects reasoning from llm_model.supports_reasoning
- Reasoning only enables when BOTH model supports it AND user enables it
- Anthropic and OpenAI Compatible nodes already had use_reasoning toggle
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The transformers library checks flash_attn version via importlib.metadata,
which fails on Windows even if the import succeeds. Now we check both
the import AND the package metadata before using flash_attention_2,
falling back to sdpa if either fails.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Convert max_tokens input to slider display mode
- Range: 512 to 4096 tokens
- Step: 128 for nice increments
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add 600s timeout to Anthropic client in both Basic and Advanced V2
generators to prevent "Streaming is strongly recommended for operations
that may take longer than 10 minutes" error.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major improvements to prompt generation:
1. NEW: prompt_mode toggle with two options:
- "Single Shot" (default): Fast single-call approach using comprehensive prompts
- "Agentic (Reasoning Models)": Deep analysis for models with reasoning capability
2. Single Shot Pipeline:
- One LLM call instead of multiple iterative calls
- Uses our component knowledge condensed into well-structured prompts
- Detail mode affects prompt depth (Standard/Detailed/Extreme)
- Auto-cleans output to remove markdown/formatting
- Much faster than iterative approach
3. Agentic Mode Validation:
- Automatically falls back to Single Shot if model doesn't support reasoning
- Clear warning when mismatch detected
- Requires reasoning-capable models (Claude 3.5, o1, etc.)
4. [DEPRECATED] Iterative pipeline:
- Multi-step looping approach marked as deprecated
- Kept for backward compatibility but no longer used by default
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds new QwenVL provider as alternative to GGUF:
- Uses HuggingFace transformers with BitsAndBytes quantization
- No llama-cpp-python required (easier cross-platform installation)
- Supports Qwen3-VL (2B/4B/8B) and Qwen2.5-VL (3B/7B) models
- Auto-downloads models from HuggingFace on first use
- Model caching option to keep loaded between runs
Changes:
- New llm_providers/sid_qwenvl_llm.py with QwenVLClient class
- Mark SID_GGUF_LLM as deprecated in favor of QwenVL
- Add qwenvl provider routing in generator _get_client
- Update template_factory.py for qwenvl provider
- Add transformers and accelerate to dependencies
- bitsandbytes as optional dependency for quantization
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
