# ComfyUI-LLM-Embeder

Local LLM chat nodes for ComfyUI, with a clean handoff path to downstream prompt optimization.

## Features
- Multi-turn chat with Ollama `/api/chat`
- Optional Hugging Face Inference API support via LLM Config
- OpenAI-compatible chat support (local servers / OpenAI / DeepSeek / Qwen) via custom `base_url`
- Anthropic Claude support via custom `base_url`
- Session memory by `session_id` (in-memory only)
- Optional `system_prompt`
- Scrollable history in a modal viewer
- `deliver_to_optimizer` action outputs the latest assistant reply only
- Auto-clear input after send (toggle)

## Nodes

### Chat (LLM)
Inputs:
- `model_name`: model name or model id for the selected provider
- `base_url`: provider base URL (ignored when provider is `huggingface`)
- `user_message`: user input text
- `action`: `send` / `regenerate` / `clear` / `deliver_to_optimizer`
- `session_id`: conversation id
- `system_prompt`: system message
- `refresh_session`: reset history for the session id
- `auto_clear_input`: clear input after a successful send
- `llm_config`: optional config from LLM Config node

Outputs:
- `assistant_response`: only non-empty when `action=deliver_to_optimizer`
- `readable_history`: full readable transcript

Behavior:
- `send`: appends user input, calls the provider, updates history
- `regenerate`: drops the last assistant turn, calls the provider again
- `clear`: resets history (keeps system prompt if set)
- `deliver_to_optimizer`: does not call the provider; returns the latest assistant reply from history

### LLM Config
Outputs a `LLM_CONFIG` struct that can be shared across Chat nodes.

Fields:
- `provider`: `ollama`, `openai_compatible`, `huggingface`, `openai`, `deepseek`, `qwen`, `claude`
- `base_url`: provider base URL (see sections below)
- `model_name`: model name or model id (provider specific)
- `temperature`
- `top_p`
- `max_new_tokens`
- `hf_token`: Hugging Face API token (recommended)
- `hf_api_url`: optional override for Hugging Face Inference API endpoint
- `api_key`: API key for OpenAI-compatible or Anthropic providers

### Chat History Viewer
Shows `readable_history` and provides an "Open History" modal for full scroll.

## Hugging Face Inference API
When `provider=huggingface`, the Chat node will call:
`https://api-inference.huggingface.co/models/{model_name}`

Notes:
- Many public models still require an HF token.
- Chat history is converted into a single prompt before the API call.

## OpenAI-Compatible API
When `provider=openai_compatible|openai|deepseek|qwen`, the Chat node will call:
`{base_url}/chat/completions`

Notes:
- Use an OpenAI-compatible base URL that includes `/v1`, or enter the full `/v1/chat/completions` URL.
- Set `api_key` when required by the server; it can remain empty for local servers without authentication.
- `model_name` should be the provider's model id.

### Local OpenAI-Compatible Servers
Select `provider=openai_compatible` in LLM Config and use the base URL exposed by the local server:

| Server | Example `base_url` |
| --- | --- |
| LM Studio | `http://127.0.0.1:1234/v1` |
| llama.cpp (`llama-server`) | `http://127.0.0.1:8080/v1` |
| LMDeploy | `http://127.0.0.1:23333/v1` |

Use the exact model id reported by the server as `model_name`. Ports may differ if the server was started with custom settings.

## Anthropic Claude API
When `provider=claude`, the Chat node will call:
`{base_url}/v1/messages`

Notes:
- Use an Anthropic-compatible base URL.
- Set `api_key` in LLM Config.

## Install
1) Place this folder under `ComfyUI/custom_nodes/`
2) Restart ComfyUI
3) Add nodes:
   - `Chat (LLM)`
   - `LLM Config`
   - `Chat History Viewer`

## Usage
1) Create `Chat (LLM)` and set `base_url` / `model_name`
2) (Optional) Add `LLM Config` to select provider and set `api_key`
3) Set `action=send`, click Execute to chat
4) When satisfied, set `action=deliver_to_optimizer` and Execute
5) Use `assistant_response` output to feed your optimizer node

## Screenshot
![ComfyUI node graph example](LLM_embeder_nodes.png)

Example graph: `LLM Config` drives `Chat (LLM)` via `llm_config`, and outputs route to a `text` node plus `Chat History Viewer` for full history review.

## Example Workflow
![Example ComfyUI workflow](Example%20workflow.png)

Drag this image into ComfyUI to load and run the workflow.

## API
Frontend calls POST `/chat_optimize/chat` and receives:
- `assistant_response`
- `readable_history`

## Notes
- History is in memory only; refresh or restart clears it.
- `assistant_response` is empty for all actions except `deliver_to_optimizer`.
- If new fields do not appear, restart ComfyUI and re-add the node.

## Contact
For questions or collaboration, email: `kanglesun@gmail.com`.

## License
MIT. See `LICENSE`.