Revise README for Remote CLIP Loader

Updated README to enhance clarity and detail about the Remote CLIP functionality, including setup instructions, node descriptions, and usage examples.
This commit is contained in:
nyueki
2026-01-09 13:40:54 -05:00
committed by GitHub
parent 6bcfc0f5a2
commit f90ce76fba
+135 -24
View File
@@ -1,31 +1,142 @@
# ComfyUI Remote CLIP Loader
# 🌐 Remote CLIP for ComfyUI
A small ComfyUI custom node set to offload CLIP encoding to a remote machine.
Run **CLIP text encoding on a different machine** and use it seamlessly inside **ComfyUI** 🚀
Perfect for offloading heavy CLIP models to another PC or server.
Motivation
I created this because I have a PC with 16GB VRAM and some large models must unload and reload the text encoder during generation, which causes inefficient generation times. I decided to run CLIP on my Mac as a remote encoder and keep my PC as the image generator.
---
What this does
- Runs a simple TCP worker that accepts prompt encode requests and returns tensors (`cond`, `pooled`).
- Worker code and client loader are in [__init__.py](__init__.py).
- Key symbols: [`RemoteCLIPWorker`](__init__.py), [`RemoteCLIPWorker.start_worker`](__init__.py), [`RemoteCLIPLoader`](__init__.py), [`RemoteCLIPLoader.load_remote`](__init__.py), [`RemoteCLIPProxy`](__init__.py), [`pack_tensors`](__init__.py), [`unpack_tensors`](__init__.py).
## ✨ What is this?
Tested with
- Flux.2 Dev
- Qwen Image
- Wan2.2
This project adds **two custom ComfyUI nodes** that let you:
Requirements
- See [requirements.txt](requirements.txt). At minimum you need PyTorch and numpy.
* 🖥️ Run a **CLIP Worker** on one machine
* 🎛️ Connect to it from another machine using a **CLIP Loader**
* 🔗 Send text prompts over the network
* 📦 Receive CLIP embeddings (`cond` + `pooled`) back in real time
Installation
1. Go to your ComfyUI installation's custom_nodes folder:
cd /path/to/ComfyUI/custom_nodes
2. Clone this repository:
git clone https://github.com/nyueki/ComfyUI-RemoteCLIPLoader.git
3. Restart ComfyUI.
Both machines run **ComfyUI**, but only **one needs the CLIP model loaded**.
Usage
- On the machine that will serve CLIP (your Mac), add the "Remote CLIP Worker" node (see [`RemoteCLIPWorker`](__init__.py)) and run it (set the CLIP model and listen port).
- On your generator PC, add the "Remote CLIP Loader" node (see [`RemoteCLIPLoader`](__init__.py)), point it to the worker's IP and port, and connect it as you would a normal CLIP model.
- The protocol and tensor packing are implemented by [`pack_tensors`](__init__.py) / [`unpack_tensors`](__init__.py).
---
## 🧠 How it works (simple view)
```
[ ComfyUI (Client) ] ---> text prompt ---> [ ComfyUI (Worker) ]
[ Remote CLIP Loader ] <--- embeddings <--- [ Remote CLIP Worker ]
```
* Uses **TCP sockets**
* Sends metadata as **JSON**
* Sends tensors as **binary blobs**
* Automatically handles tensor shapes & dtypes
---
## 🧩 Nodes Included
### 🔹 Remote CLIP Worker
🖥️ **Runs on the machine that has the CLIP model**
* Listens on a TCP port
* Receives text prompts
* Runs CLIP encoding
* Sends embeddings back to clients
**Inputs**
* `clip` → Any CLIP model
* `listen_port` → Port to listen on (default: `8002`)
---
### 🔹 Remote CLIP Loader
🎛️ **Runs on the client machine**
* Connects to the worker
* Acts like a normal CLIP object in ComfyUI
* Transparently forwards encoding requests
**Inputs**
* `worker_ip` → IP address of the worker machine
* `port` → Worker port (default: `8002`)
---
## ⚙️ Setup Instructions
### 1️⃣ Install on both machines
Clone this repository into your ComfyUI `custom_nodes` directory:
```
cd ComfyUI/custom_nodes
git clone https://github.com/nyueki/ComfyUI-RemoteCLIPLoader.git
```
Restart ComfyUI on **both machines**.
---
### 2️⃣ Start the Worker (Machine A)
1. Add **Remote CLIP Worker** node
2. Connect a CLIP model
3. Choose a port (e.g. `8002`)
4. Queue the workflow
✅ The worker is now listening for connections
---
### 3️⃣ Connect from Client (Machine B)
1. Add **Remote CLIP Loader** node
2. Enter the worker’s IP address
3. Match the port number
4. Use it anywhere a CLIP node is expected 🎉
---
## 📦 What gets transmitted?
* 📝 Prompt text
* 🔢 Tokenization options
* 🧠 CLIP embeddings:
* `cond` → `(1, 77, 768)`
* `pooled` → `(1, 768)`
All tensors are:
* Converted to CPU
* Sent as raw bytes
* Reconstructed safely on the client
---
## 🛡️ Notes & Tips
* 🔌 Make sure the port is open on the worker machine
* 🏠 Best used on local networks (LAN)
* ⚠️ No encryption — don’t expose to the public internet
* 🧪 Designed for simplicity & reliability
---
## 🧰 Category in ComfyUI
```
Remote CLIP
```
---
## ❤️ Why use this?
* Save VRAM on your main machine
* Share one powerful CLIP server
* Experiment with distributed ComfyUI setups
* Clean, minimal, no external dependencies