nodes/reeoi-ComfyUI-WebROI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add files via upload

Browse Source
This commit is contained in:
reeoi
2025-11-25 16:36:17 +08:00
committed by GitHub
parent 6eda12598a
commit b4166d6561
5 changed files with 438 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LICENSE
+17
View File
@@ -0,0 +1,17 @@
MIT License
Copyright (c) [2025] [reeoi]
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
+65
View File
@@ -0,0 +1,65 @@
# ComfyUI-WebROI Capture 🖥️✂️
<div align="center">
[🇺🇸 English](#english) | [🇨🇳 中文](#chinese)
**Capture screen, windows, or tabs directly in your browser and send them to ComfyUI.**
**无需服务器显卡驱动，直接在浏览器内截取屏幕、窗口或标签页，并发送给 ComfyUI。**
</div>
---
<a name="english"></a>
## 🇺🇸 English
**ComfyUI-WebROI** is a custom node that enables client-side screen capturing directly within the browser. It supports **Region of Interest (ROI)** selection, allowing you to crop specific parts of your screen before sending them to the workflow.
Since the capture happens locally in your browser and data is sent via Base64, it is **100% Cloud Compatible** (AutoDL, RunPod, Colab, AWS) and requires **no GUI or display drivers** on the server side.
### ✨ Key Features
* **🖥️ Client-Side Capture**: Uses the browser's native `navigator.mediaDevices` API. No external software (like OBS) needed.
* **🖱️ Interactive ROI Cropping**:
* **Draw**: Drag your mouse on the node preview to draw a selection box (Yellow).
* **Lock**: Release mouse to lock the crop area (Red). Only this area is sent to the backend.
* **Reset**: Double-click or press the `RESET` button to clear the selection.
* **📐 Resizable UI**: The node is fully resizable. You can shrink it to save space on your canvas, and the preview will adapt automatically.
* **☁️ Cloud Ready**: Works perfectly on headless cloud servers. As long as you can see the ComfyUI web page, you can stream your screen.
* **⚡ Real-time Workflow**: Compatible with ComfyUI's `Auto Queue` for real-time Screen-to-Image generation.
### ⚠️ CRITICAL: HTTPS Requirement
**If you are using a remote server (Cloud), you MUST access ComfyUI via HTTPS.**
Modern browsers **block** screen sharing APIs on insecure (HTTP) connections.
* ✅ `http://localhost` or `http://127.0.0.1` -> **Works** (Local)
* ✅ `https://your-cloud-url.com` -> **Works** (Cloud with SSL)
* ❌ `http://192.168.x.x` or `http://xx.xx.xx.xx:8188` -> **BLOCKED**
**How to fix on Cloud:**
1. Use the **Cloudflare Tunnel** or **Ngrok** to get an `https` link.
2. Use the official Proxy Link provided by services like RunPod.
3. (Testing only) Enable `chrome://flags/#unsafely-treat-insecure-origin-as-secure` in Chrome and add your HTTP URL.
### 📦 Installation
1. Navigate to your `custom_nodes` folder:
```bash
cd ComfyUI/custom_nodes/
2. Clone this repository：
git clone https://github.com/reeoi/ComfyUI-WebROI.git
3. Restart ComfyUI.
📖 Usage
1. Add Node: Double-click and search for WebROICapture.
2. Start Capture: Click the 🖥️ START button on the node. Your browser will ask which screen/window to share.
3. Select Region (Optional):
1)Drag on the video preview to draw a box.
2)The generated image will only contain the selected area.
3)Double-click to reset to full view.
4.Generate: Connect the IMAGE output to your workflow and click Queue Prompt.
5.Real-time: Check Auto Queue in the ComfyUI menu to stream frames continuously.
__init__.py
+6
View File
@@ -0,0 +1,6 @@
from .web_roi import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
# 这一行告诉 ComfyUI：前端文件在当前目录下的 js 文件夹里
WEB_DIRECTORY = "./js"
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', 'WEB_DIRECTORY']
js/web_roi.js
+286
View File
@@ -0,0 +1,286 @@
import { app } from "../../scripts/app.js";
app.registerExtension({
name: "ComfyUI.WebROI",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name !== "WebROICapture") return;
console.log("✅ WebROI: V5 Mouse Selection Loaded");
// ============================================================
// 1. 绘制函数 (负责画视频、画选框、截图)
// ============================================================
nodeType.prototype.onDrawForeground = function(ctx) {
if (this.flags.collapsed) return;
// 绘制左上角红点 (运行指示器)
ctx.fillStyle = "red";
ctx.beginPath();
ctx.arc(10, 10, 5, 0, 2 * Math.PI);
ctx.fill();
const roi = this.webRoiCtx;
if (!roi) return;
// 绘制状态文字
if (roi.statusText) {
ctx.fillStyle = "white";
ctx.font = "14px Arial";
ctx.fillText(roi.statusText, 20, 50);
}
// 如果没有视频，就不继续了
if (!roi.isSharing || !roi.video || roi.video.readyState < 2) return;
try {
// --- A. 计算显示区域 ---
const nodeWidth = this.size[0];
const contentWidth = nodeWidth - 20; // 左右留边
const aspectRatio = roi.video.videoHeight / roi.video.videoWidth;
const contentHeight = contentWidth * aspectRatio;
// 记录视频在节点上的显示位置 (供鼠标事件使用)
const drawX = 10;
const drawY = 60; // 避开按钮
roi.renderRect = { x: drawX, y: drawY, w: contentWidth, h: contentHeight };
// --- B. 绘制视频到底层 ---
ctx.drawImage(roi.video, drawX, drawY, contentWidth, contentHeight);
// --- C. 绘制选框 (UI) ---
// 计算当前的选框数据 (相对于节点坐标)
let selRect = null;
// 如果正在拖拽，优先显示拖拽框
if (roi.dragStart && roi.dragCurrent) {
const x = Math.min(roi.dragStart[0], roi.dragCurrent[0]);
const y = Math.min(roi.dragStart[1], roi.dragCurrent[1]);
const w = Math.abs(roi.dragCurrent[0] - roi.dragStart[0]);
const h = Math.abs(roi.dragCurrent[1] - roi.dragStart[1]);
selRect = { x, y, w, h };
// 拖拽时画黄色虚线
ctx.strokeStyle = "yellow";
ctx.lineWidth = 2;
ctx.setLineDash([5, 5]);
ctx.strokeRect(x + drawX, y + drawY, w, h);
ctx.setLineDash([]);
}
// 如果有已锁定的选区
else if (roi.selection) {
// selection 存储的是相对比例 (0.0 - 1.0)，需要转回像素
const x = roi.selection.x * contentWidth;
const y = roi.selection.y * contentHeight;
const w = roi.selection.w * contentWidth;
const h = roi.selection.h * contentHeight;
// 锁定后画红色实线
ctx.strokeStyle = "#ff0000";
ctx.lineWidth = 3;
ctx.strokeRect(x + drawX, y + drawY, w, h);
}
// --- D. 截取数据 (Crop & Send) ---
// 只有当尺寸变化或每一帧都更新时执行
const vidW = roi.video.videoWidth;
const vidH = roi.video.videoHeight;
// 确定源坐标 (Source X, Y, W, H)
let sx = 0, sy = 0, sW = vidW, sH = vidH;
if (roi.selection) {
sx = Math.floor(roi.selection.x * vidW);
sy = Math.floor(roi.selection.y * vidH);
sW = Math.floor(roi.selection.w * vidW);
sH = Math.floor(roi.selection.h * vidH);
}
// 安全检查，防止超出边界
if (sW <= 0) sW = 1; if (sH <= 0) sH = 1;
// 调整画布尺寸以适应裁剪后的大小
if (roi.cropCanvas.width !== sW || roi.cropCanvas.height !== sH) {
roi.cropCanvas.width = sW;
roi.cropCanvas.height = sH;
}
// 执行裁剪绘制: drawImage(img, sx, sy, sw, sh, dx, dy, dw, dh)
roi.cropCtx.drawImage(roi.video, sx, sy, sW, sH, 0, 0, sW, sH);
// 更新数据给 Python
const widget = this.widgets.find(w => w.name === "image_data");
if (widget) {
widget.value = roi.cropCanvas.toDataURL("image/jpeg", 0.7);
}
} catch (e) {
console.error("Draw error:", e);
}
};
// ============================================================
// 2. 鼠标交互逻辑
// ============================================================
// 按下鼠标
nodeType.prototype.onMouseDown = function(e, pos) {
const roi = this.webRoiCtx;
if (!roi || !roi.renderRect) return;
// 检查点击是否在视频区域内
const r = roi.renderRect;
const mx = pos[0];
const my = pos[1];
if (mx >= r.x && mx <= r.x + r.w && my >= r.y && my <= r.y + r.h) {
// 记录相对于视频左上角的坐标
roi.dragStart = [mx - r.x, my - r.y];
roi.dragCurrent = [mx - r.x, my - r.y];
roi.selection = null; // 清除旧选区
return true; // 捕获事件，防止拖动节点
}
};
// 移动鼠标
nodeType.prototype.onMouseMove = function(e, pos) {
const roi = this.webRoiCtx;
if (!roi || !roi.dragStart) return;
const r = roi.renderRect;
// 限制坐标在视频范围内
let currX = pos[0] - r.x;
let currY = pos[1] - r.y;
// Clamp
currX = Math.max(0, Math.min(currX, r.w));
currY = Math.max(0, Math.min(currY, r.h));
roi.dragCurrent = [currX, currY];
};
// 松开鼠标
nodeType.prototype.onMouseUp = function(e, pos) {
const roi = this.webRoiCtx;
if (!roi || !roi.dragStart) return;
// 计算最终选区
const x1 = roi.dragStart[0];
const y1 = roi.dragStart[1];
const x2 = roi.dragCurrent[0];
const y2 = roi.dragCurrent[1];
const x = Math.min(x1, x2);
const y = Math.min(y1, y2);
const w = Math.abs(x1 - x2);
const h = Math.abs(y1 - y2);
// 如果选区太小（比如只是点了一下），视为取消选区
if (w > 10 && h > 10) {
// 保存为相对比例 (0.0 - 1.0)，这样分辨率改变也不怕
roi.selection = {
x: x / roi.renderRect.w,
y: y / roi.renderRect.h,
w: w / roi.renderRect.w,
h: h / roi.renderRect.h
};
} else {
roi.selection = null;
}
roi.dragStart = null;
roi.dragCurrent = null;
};
// 双击重置
nodeType.prototype.onDblClick = function(e, pos) {
const roi = this.webRoiCtx;
if (roi) {
roi.selection = null;
console.log("Selection Reset");
}
}
// ============================================================
// 3. 强制高度计算 (同 V4)
// ============================================================
const origComputeSize = nodeType.prototype.computeSize;
nodeType.prototype.computeSize = function() {
if (this.webRoiCtx && this.webRoiCtx.isSharing && this.webRoiCtx.video) {
const vid = this.webRoiCtx.video;
if (vid.videoWidth > 0) {
const ar = vid.videoHeight / vid.videoWidth;
return [this.size[0], (this.size[0] - 20) * ar + 100];
}
}
return origComputeSize ? origComputeSize.apply(this, arguments) : [200, 100];
};
// ============================================================
// 4. 初始化与按钮
// ============================================================
const onNodeCreated = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function () {
const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
this.webRoiCtx = {
video: document.createElement("video"),
cropCanvas: document.createElement("canvas"),
cropCtx: null,
statusText: "Click Start ->",
isSharing: false,
renderRect: null, // 视频在节点上的位置
selection: null, // 最终选区 {x,y,w,h} (比例 0-1)
dragStart: null, // 拖拽起点
dragCurrent: null // 拖拽终点
};
const roi = this.webRoiCtx;
roi.cropCtx = roi.cropCanvas.getContext("2d");
roi.video.autoplay = true;
roi.video.muted = true;
roi.video.style.display = "none";
document.body.appendChild(roi.video);
// 按钮 1: 开始捕捉
this.addWidget("button", "🖥️ START", null, () => {
startScreenShare(this, roi);
});
// 按钮 2: 重置选区
this.addWidget("button", "❌ RESET ROI", null, () => {
roi.selection = null;
});
setTimeout(() => {
const w = this.widgets.find(w => w.name === "image_data");
if (w) w.type = "hidden";
this.setSize([360, 150]);
}, 100);
return r;
};
async function startScreenShare(node, roi) {
try {
const stream = await navigator.mediaDevices.getDisplayMedia({
video: { cursor: "always" }, audio: false
});
roi.video.srcObject = stream;
roi.video.onloadedmetadata = () => {
roi.video.play();
roi.statusText = "";
roi.isSharing = true;
node.setSize(node.computeSize());
renderLoop();
};
} catch (e) {
roi.statusText = "Error: " + e.message;
}
}
function renderLoop() {
app.graph.setDirtyCanvas(true, false);
requestAnimationFrame(renderLoop);
}
}
});
web_roi.py
+64
View File
@@ -0,0 +1,64 @@
import torch
import numpy as np
from PIL import Image
import base64
from io import BytesIO
import re
class WebROICapture:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {
"required": {
# 必须有这个输入项，前端JS才会把图片数据塞进来
"image_data": ("STRING", {"default": "", "multiline": True}),
},
"optional": {
# 用作触发器，每次数字变化时触发
"trigger_always": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "forceInput": True}),
}
}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("image",)
FUNCTION = "capture_screen"
CATEGORY = "WebROI"
def capture_screen(self, image_data, trigger_always=0):
try:
# 1. 检查数据是否为空
if not image_data or "," not in image_data:
# 返回一张黑图防止报错
print("⚠️ WebROI: Waiting for image data...")
empty_img = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
return (empty_img,)
# 2. 解码 Base64
base64_data = image_data.split(",")[1]
image_bytes = base64.b64decode(base64_data)
img = Image.open(BytesIO(image_bytes))
# 3. 转换为 ComfyUI 格式 (Tensor)
img = img.convert("RGB")
img_np = np.array(img).astype(np.float32) / 255.0
img_tensor = torch.from_numpy(img_np)[None,]
return (img_tensor,)
except Exception as e:
print(f"❌ WebROI Error: {e}")
empty_img = torch.zeros((1, 512, 512, 3), dtype=torch.float32)
return (empty_img,)
# 节点映射名称，JS 必须用这个名字 "WebROICapture" 才能找到它
NODE_CLASS_MAPPINGS = {
"WebROICapture": WebROICapture
}
NODE_DISPLAY_NAME_MAPPINGS = {
"WebROICapture": "🖥️ Web ROI Capture"
}