nodes/arcum42-ComfyUI_SageUtils

Fork 0

Files

T

Shanoah Alkire 73a604ea09 Bring docs more in sync with local files.

2025-10-14 14:44:48 -07:00

11 KiB

Raw Blame History

LLM Preset API Documentation

This document describes the backend API endpoints for programmatically using LLM presets without going through the UI.

Endpoints

1. Get All Presets

Get a complete list of all available LLM presets (built-in + custom).

Endpoint: GET /sage_llm/presets/all

Response:

{
    "success": true,
    "data": {
        "presets": {
            "descriptive_prompt": {
                "name": "Descriptive Prompt",
                "description": "Generate detailed image descriptions",
                "provider": "ollama",
                "model": "gemma3:12b",
                "category": "description",
                "isBuiltin": true,
                "promptTemplate": "description/Descriptive Prompt",
                "systemPrompt": "e621_prompt_generator",
                "settings": {
                    "temperature": 0.7,
                    "seed": -1,
                    "maxTokens": 512,
                    "keepAlive": 300,
                    "includeHistory": false
                }
            },
            "e621_description": {
                "name": "E621 Image Description",
                "description": "Generate E621-style detailed image descriptions",
                "provider": "ollama",
                "model": "gemma3:12b",
                "category": "description",
                "isBuiltin": true,
                "promptTemplate": "description/Descriptive Prompt",
                "systemPrompt": "e621_prompt_generator",
                "settings": {
                    "temperature": 0.8,
                    "seed": -1,
                    "maxTokens": 1024,
                    "keepAlive": 300,
                    "includeHistory": false
                }
            },
            "casual_chat": {
                "name": "Casual Chat",
                "description": "Friendly conversational assistant",
                "provider": "ollama",
                "model": null,
                "category": "chat",
                "isBuiltin": true,
                "promptTemplate": "",
                "systemPrompt": "default",
                "settings": {
                    "temperature": 0.9,
                    "seed": -1,
                    "maxTokens": 1024,
                    "keepAlive": 300,
                    "includeHistory": true,
                    "maxHistoryMessages": 10
                }
            }
        }
    }
}

Python Example:

import requests

response = requests.get('http://127.0.0.1:8188/sage_llm/presets/all')
data = response.json()

if data['success']:
    presets = data['data']['presets']
    
    # List all preset names
    for preset_id, preset in presets.items():
        print(f"{preset_id}: {preset['name']}")
        print(f"  Model: {preset['model']}")
        print(f"  Category: {preset['category']}")
        print()

2. Generate with Preset and Image

Use a preset to generate a response with an image, all programmatically.

Endpoint: POST /sage_llm/presets/generate_with_image

Request Body:

{
    "preset_id": "descriptive_prompt",
    "images": ["base64_encoded_image_data"],
    "prompt_override": "Optional: custom prompt instead of preset's template",
    "system_prompt_override": "Optional: custom system prompt",
    "settings_override": {
        "temperature": 0.9,
        "seed": 12345
    }
}

Parameters:

preset_id (required): ID of the preset to use
images (required): Array of base64-encoded images
prompt_override (optional): Override the preset's prompt template
system_prompt_override (optional): Override the preset's system prompt
settings_override (optional): Override specific settings (temperature, seed, etc.)

Response:

{
    "success": true,
    "data": {
        "response": "Generated description text...",
        "preset_used": "descriptive_prompt",
        "provider": "ollama",
        "model": "gemma3:12b"
    }
}

Python Example:

import requests
import base64
from pathlib import Path

# Read and encode image
image_path = Path("path/to/image.png")
with open(image_path, 'rb') as f:
    image_data = base64.b64encode(f.read()).decode('utf-8')

# Generate description using preset
response = requests.post(
    'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
    json={
        'preset_id': 'descriptive_prompt',
        'images': [image_data]
    }
)

data = response.json()
if data['success']:
    print(f"Generated description: {data['data']['response']}")
    print(f"Used model: {data['data']['model']}")

Python Example with Overrides:

import requests
import base64

# Read image
with open('image.png', 'rb') as f:
    image_b64 = base64.b64encode(f.read()).decode('utf-8')

# Use preset but override specific settings
response = requests.post(
    'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
    json={
        'preset_id': 'e621_description',
        'images': [image_b64],
        'prompt_override': 'Analyze this image and describe the main subjects.',
        'settings_override': {
            'temperature': 0.5,  # More deterministic
            'seed': 42  # Reproducible
        }
    }
)

result = response.json()
if result['success']:
    description = result['data']['response']
    print(description)

Use Cases

1. Batch Image Processing

import requests
import base64
from pathlib import Path

def describe_images_batch(image_paths, preset_id='descriptive_prompt'):
    """Process multiple images with a preset."""
    results = []
    
    for img_path in image_paths:
        with open(img_path, 'rb') as f:
            img_b64 = base64.b64encode(f.read()).decode('utf-8')
        
        response = requests.post(
            'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
            json={
                'preset_id': preset_id,
                'images': [img_b64]
            }
        )
        
        if response.json()['success']:
            results.append({
                'image': img_path,
                'description': response.json()['data']['response']
            })
    
    return results

# Use it
images = list(Path('images/').glob('*.png'))
descriptions = describe_images_batch(images)

for result in descriptions:
    print(f"{result['image']}: {result['description']}\n")

2. Custom Node Integration

class LLMPresetDescriptionNode:
    """ComfyUI node that uses LLM presets."""
    
    @classmethod
    def INPUT_TYPES(cls):
        # Get available presets
        import requests
        response = requests.get('http://127.0.0.1:8188/sage_llm/presets/all')
        presets = []
        if response.json()['success']:
            presets = list(response.json()['data']['presets'].keys())
        
        return {
            "required": {
                "image": ("IMAGE",),
                "preset": (presets,),
            }
        }
    
    RETURN_TYPES = ("STRING",)
    FUNCTION = "generate"
    CATEGORY = "sage_utils/llm"
    
    def generate(self, image, preset):
        import requests
        import base64
        import numpy as np
        from PIL import Image
        import io
        
        # Convert ComfyUI image tensor to base64
        i = 255. * image[0].cpu().numpy()
        img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
        
        buffer = io.BytesIO()
        img.save(buffer, format='PNG')
        img_b64 = base64.b64encode(buffer.getvalue()).decode('utf-8')
        
        # Generate using preset
        response = requests.post(
            'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
            json={
                'preset_id': preset,
                'images': [img_b64]
            }
        )
        
        if response.json()['success']:
            return (response.json()['data']['response'],)
        else:
            return ("Error generating description",)

3. Workflow Automation

def auto_tag_images(folder_path, output_file='tags.json'):
    """Automatically tag all images in a folder."""
    import requests
    import base64
    import json
    from pathlib import Path
    
    tags = {}
    
    for img_path in Path(folder_path).glob('*.png'):
        with open(img_path, 'rb') as f:
            img_b64 = base64.b64encode(f.read()).decode('utf-8')
        
        # Use e621_description preset for detailed tags
        response = requests.post(
            'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
            json={
                'preset_id': 'e621_description',
                'images': [img_b64]
            }
        )
        
        if response.json()['success']:
            tags[str(img_path)] = response.json()['data']['response']
    
    # Save to file
    with open(output_file, 'w') as f:
        json.dump(tags, f, indent=2)
    
    return tags

# Use it
tags = auto_tag_images('output/images')
print(f"Tagged {len(tags)} images")

Error Handling

Both endpoints return errors in a consistent format:

{
    "success": false,
    "error": "Error message here"
}

Common errors:

400: Missing required parameters or invalid data
404: Preset not found
500: Server error (LLM unavailable, file error, etc.)
503: LLM provider (Ollama/LM Studio) not available

Example Error Handling:

import requests

response = requests.post(
    'http://127.0.0.1:8188/sage_llm/presets/generate_with_image',
    json={'preset_id': 'invalid_preset', 'images': ['...']},
    timeout=60
)

data = response.json()

if data['success']:
    print(data['data']['response'])
else:
    print(f"Error: {data['error']}")
    
    # Check status code
    if response.status_code == 404:
        print("Preset not found")
    elif response.status_code == 503:
        print("LLM provider unavailable")

Notes

Image Format: Images must be base64-encoded. Any image format supported by PIL can be used.
Multiple Images: You can pass multiple images in the images array. This is useful for vision models that support multiple image inputs.
Preset Overrides: User-customized versions of built-in presets automatically take precedence.
System Prompts: The API handles loading system prompts from both built-in assets and user custom prompts.
Prompt Templates: If a preset specifies a promptTemplate, it will be loaded from the LLM prompts configuration unless overridden.
Keep Alive: The keepAlive setting controls how long the model stays loaded in memory (in seconds). Convert to minutes for Ollama.
Async Operations: For better performance in production, consider using async HTTP clients like aiohttp or httpx.

11 KiB Raw Blame History