56 lines
2.2 KiB
Markdown
56 lines
2.2 KiB
Markdown
# Yet Another LLM Node (for ComfyUI)
|
|
|
|
Yet another set of LLM nodes for [ComfyUI](https://github.com/comfyanonymous/ComfyUI). This one only supports OpenAI-like APIs, but of course can be used with local LLM providers such as [llama.cpp](https://github.com/ggerganov/llama.cpp) (and, I believe, ollama and LMStudio, among others).
|
|
|
|
This project mainly served as an exercise in creating ComfyUI nodes.
|
|
|
|
|
|
|
|
(the "Show Text" node is from the very useful https://github.com/pythongosssss/ComfyUI-Custom-Scripts)
|
|
|
|
## Features
|
|
|
|
* Connection profiles are defined externally in `models.yaml`, optionally pulling API URLs & API keys from environment variables.
|
|
|
|
Since they're externally defined, there's no chance for your precious API keys to leak via workflow metadata.
|
|
* Only uses the chat completion API endpoint. No messing around with prompt templates.
|
|
* Can set a system prompt.
|
|
* The usual set of LLM samplers (temperature, top-k, top-p, min-p)
|
|
|
|
For llama.cpp at least, the order you chain them together affects their order of application. (Other llama.cpp-derived providers might also respect the `samplers` option.)
|
|
|
|
Also note that the OpenAI API spec only officially supports temperature and top-p, so what's actually supported will depend on your provider.
|
|
* Seed is exposed and configurable, allowing some degree of determinism.
|
|
|
|
Aside from adding more samplers (as requested/as I need them), I consider this project feature complete.
|
|
|
|
Though don't be surprised if I start adding RAG features or something in the future.
|
|
|
|
## Installation
|
|
|
|
Clone this project into your `custom_nodes` directory.
|
|
|
|
Then within the `ComfyUI-YALLM-node` directory:
|
|
|
|
cp models.yaml.example models.yaml
|
|
|
|
and edit `models.yaml` if you would like to use more than just the `http://localhost:8080/v1` endpoint.
|
|
|
|
Finally, install the [openai](https://pypi.org/project/openai/) Python package (currently the only dependency):
|
|
|
|
pip install openai
|
|
|
|
or
|
|
|
|
pip install -r requirements.txt
|
|
|
|
or
|
|
|
|
path/to/ComfyUI/python_embeded/python -m pip install -r requirements.txt
|
|
|
|
Just make sure `pip`/`python` is the same one used for your ComfyUI installation.
|
|
|
|
## License
|
|
|
|
Licensed under [BSD-2-Clause-Patent](https://opensource.org/license/bsdpluspatent).
|