nodes/lks-ai-anynode
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'main' of github.com-lks:lks-ai/anynode

Browse Source
This commit is contained in:
holonic
2024-06-07 01:50:04 +01:00
parent 241401114c f60c3bab31
commit d8fc8b1f75
1 changed files with 28 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+28 -7
View File
@@ -14,7 +14,7 @@ A ComfyUI Node that uses the power of LLMs to do anything with your input to mak
### [Join our Discord](https://discord.gg/RFpe6gsK5x)
## Install: Update (We just got on ComfyUI Manager!)
## Install
1. Clone this repository into `comfy/custom_nodes` *or* Just search for `AnyNode` on ComfyUI Manager
2. If you're using openAI API, follow the OpenAI instructions
@@ -55,21 +55,39 @@ AnyNode codes a python function based on your request and whatever input you con
![image](https://github.com/lks-ai/anynode/assets/163685473/1245aa94-fa4d-4490-a3f4-5e8b9918ca28)
**Warning**: Because of the ability to link *ANY* node, you have to make sure it nails the output.
## Update: It can make you a sandwich
![Courtesy of Synthetic Ape](https://github.com/lks-ai/anynode/assets/163685473/fc40a7fe-61d4-4347-aeec-80d5b34ddf4f)
*Courtesy of [synthetic ape](https://www.tiktok.com/@synthetic.ape)*
## Caveats
**Warning**: Because of the ability to link *ANY* node, you can crash ComfyUI if you are not careful.
### 🛡️ Security Features
You shouldn't trust an LLM with your computer, and we don't either.
**Code Sanizitzer**
Every piece of code that the LLM outputs [goes through a sanitizer](https://github.com/lks-ai/anynode/blob/d2d466bdcf6ef162aef503b3c79e135b7a33c349/nodes/utils.py#L149) before being allowed to be loaded into the environment or executed. You will see errors about dangerous code... that's the sanitizer.
**No Internet, No Files, No Command Line**
As a safety feature, AnyNode does not have the ability to generate functions that browse the internet or touch the files on your computer. If you need to load something into comfy or get stuff from the internet, there are plenty of loader nodes available in popular node packs on Manager.
**Curated Imports**
We only let AnyNode use libraries from the list of [Allowed Imports](https://github.com/lks-ai/anynode/blob/d2d466bdcf6ef162aef503b3c79e135b7a33c349/nodes/any.py#L120). Anything else will not even be within the function's runtime environment and will give you an error. This is a feature. If you want libraries you don't see in that list to be added to AnyNode, let us know on the Discord or open an Issue.
**Note**: AnyNode can use the openai and google generativeAI libraries in the functions it generates, so you can ask it to use the latest from OpenAI by pasting an example from their API and get it to stream a TTS audio file to your computer, that is a supported library and it's fine.
## 🤔 Caveats
- I have no idea how far you can take this nor it's limits
- LLMs can't read your mind. To make complex stuff in one node you'd have to know a bit about programming
- The smaller the LLM you use to code your nodes, the less coding skills it might have
- Right now you can only see code the LLM generates in the console
- Can't make a sandwich
- ~~Can't make a sandwich~~
## Strengths
## 💪 Strengths
- Use OpenAI `AnyNode 🍄`, Local LLMs `AnyNode 🍄 (Local LLM)`, Gemini `AnyNode 🍄 (Gemini)`
- You can use as many of these as you want in your workflow creating possibly complex node groups
- Really great at single purpose nodes
- Uses OpenAI API for simple access to the latest and greatest in generation models
- Technically you could point this at vLLM or Ollama for you LocalLLM fans
- Technically you could point this at vLLM. LM Studio or Ollama for you LocalLLM fans
- Can use most of the popular python libraries and most standard like (numpy, torch, collections, re)
- Ability to make more complex nodes that use inputs like MODEL, VAE and CLIP with input type awareness
- Error Mitigation: Auto-correct errors it made in code (just press `Queue Prompt` again)
@@ -77,11 +95,14 @@ AnyNode codes a python function based on your request and whatever input you con
- Copy cool nodes you prompt is as easy as copying the workflow
- Saves generated functions registry `json` to `output/anynode` so you can bundle it with workflow
- Can make more complex functions with two optional inputs to the node.
- **IT CAN MAKE A SANDWICH!**
## Coming Soon
## 🛣️ Roadmap
- **Export to Node**: Compile a new comfy node from your AnyNode (Requires restart to use your new node)
- RAG based function storage and semantic search across comfy modules (not a pipe dream)
- Persistent data storage in the AnyNode (functions store extra data for iterative processing or persistent memory)
- Expanding [*NodeAware*](https://github.com/lks-ai/anynode/blob/main/nodes/util_nodeaware.py#L1) to include full Workspace Awareness
- Node Recommendations: AnyNode will recommend you or even load some nodes into the workflow based on your input
## Coding Errors you Might Encounter
As with any LLMs or text generating language model, when it comes to coding, it can sometimes make mistakes that it can't fix by itself even if you show it the error of it's ways. A lot of these can be mitigated by modifying your prompt. If you encounter some of the known ones, we have some prompt engineering solutions here for you.