lks-ai-anynode/README.md

AnyNode v0.1 (🍄 beta)

A ComfyUI Node that uses the power of LLMs to do anything with your input to make any type of output.

Update: Day 3 - Local LLMs Generating Nodes!

We now have an AnyNode 🍄 (Gemini) Node and our big star: The AnyNode 🍄 (Local LLM) Node. This was the most requested feature since Day 1. The classic AnyNode 🍄 will still use OpenAI directly.

• You can set each LocalLLM node to use a different local or hosted service as long as it's OpenAI compatible
• This means you can use Ollama, vLLM and any other LocalLLM server from wherever you want

Install: Update (We just got on ComfyUI Manager!)

1. Clone this repository into comfy/custom_nodes or Just search for AnyNode on ComfyUI Manager
2. If you're using openAI API, follow the openAI instructions
3. If you're using LocalLLMs API, make sure your LLM server (ollama, etc.) is running
4. Restart Comfy
5. In ComfyUI double-click and search for AnyNode or you can find it in Nodes > utils

OpenAI Instructions

1. Make sure you have the openai module installed through pip: pip install openai
2. Add your OPENAI_API_KEY variable to your Environment Variables. How to get your OpenAI API key

How it Works

1. Put in what you want the node to do with the input and output.
2. Connect it up to anything on both sides
3. Hit Queue Prompt in ComfyUI

AnyNode codes a python function based on your request and whatever input you connect to it to generate the output you requested which you can then connect to compatible nodes.

Warning: Because of the ability to link ANY node, you have to make sure it nails the output.

Caveats

• I have no idea how far you can take this nor it's limits
• LLMs can't read your mind. To make complex stuff in one node you'd have to know a bit about programming
• The smaller the LLM you use to code your nodes, the less coding skills it might have
• Right now you can only see code the LLM generates in the console
• You currently need an OpenAI API Key
• Can't make a sandwich

Strengths

• Use OpenAI AnyNode, Local LLMs AnyNode Local, Gemini AnyNode Gemini
• You can use as many of these as you want in your workflow creating possibly complex node groups
• Really great at single purpose nodes
• Uses OpenAI API for simple access to the latest and greatest in generation models
• Technically you could point this at vLLM or Ollama for you LocalLLM fans
• Can use most of the popular python libraries and most standard like (numpy, torch, collections, re)
• Ability to make more complex nodes that use inputs like MODEL, VAE and CLIP with input type awareness
• Error Mitigation: Auto-correct errors it made in code (just press Queue Prompt again)
• Incremental Code editing (the last generated function serves as example for next generation)
• Copying cool nodes you prompt is as easy as copying the workflow

Coming Soon

• Export to Node: Compile a new comfy node from your AnyNode (Requires restart to use your new node)
• Saving the generated functions in your workflows
• Multiple Inputs and outputs
• RAG based function storage and semantic search across comfy modules (not a pipe dream)
• Persistent data storage in the AnyNode (functions store extra data for iterative processing or persistent memory)

If you're still here

Let's enjoy some stuff I made while up all night!

This one, well... the prompts explain it all, but TLDR; It takes an image as input and outputs only the red channel of that image.

Here I use three AnyNodes: One to load a file, one to summarize the text in that file, and the other to just do some parsing of that text. No coding needed.

I took that Ant example a bit further and added in the normal nodes to do img2img with my color transforms from AnyNode

Here I ask for an instagram-like sepia tone filter for my AnyNode ... I titled the node Image Filter just so I can remember what it's supposed to be doing in the workflow

Let's try a much more complex description of an HSV transform, but still in plain english. And we get a node that will randomly filter HSV every time it's run!

Then I ask for a more legacy instagram filter (normally it would pop the saturation and warm the light up, which it did!)

How about a psychedelic filter?

Here I ask it to make a "sota edge detector" for the output image, and it makes me a pretty cool Sobel filter. And I pretend that I'm on the moon.