nodes/VrchStudio-comfyui-web-viewer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
c4d41cb8a6c2c3387de0faa7cc8a4acf987eeacc
VrchStudio-comfyui-web-viewer/docs/websocket_nodes.md
T

624 lines
38 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
### Node: `IMAGE WebSocket Web Viewer (Legacy) @ vrch.ai` (vrch.ai/viewer)
1. **Add the `IMAGE WebSocket Web Viewer (Lagecy) @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Image Input:**
- **`images`**: Connect the image(s) you wish to display in the WebSocket-based web viewer.
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to differentiate WebSocket connections.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Image Format:**
- **`format`**: Choose the image format for transmission, supporting **PNG** and **JPEG** (default is **JPEG**).
- **Websocket Parameters:**
- **`number_of_images`**: Set the number of images to load (default is **`4`**, range: 1-99).
- **`image_display_duration`**: Duration to display each image in milliseconds (default is **`1000`**, range: 1-10000).
- **`fade_anim_duration`**: Duration of fade animation in milliseconds (default is **`200`**, range: 1-10000).
- **`mixBlendMode`**: Set the blend mode for image transitions (default is **`none`**). Options include **`normal`**, **`multiply`**, **`screen`**, etc.
- **`enableLoop`**: Toggle whether to loop playback of images (default is **`True`**).
- **`enableUpdateOnEnd`**: Toggle whether to update the image cache only at the end of playback (default is **`False`**).
- **Server Messages:** Save and send server messages to its web page viewer.
- **Save Settings:** Toggle whether to save the websocket settings to a JSON file. When enabled, sends settings via WebSocket.
- **Window Dimensions:**
- **`window_width`**: Set the width of the web viewer window (default is **1280**).
- **`window_height`**: Set the height of the web viewer window (default is **960**).
- **Show URL:**
- **`show_url`**: Toggle the display of the constructed URL in the interface. When enabled, the **`url`** field becomes visible.
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
- **Extra Parameters:**
- **`extra_params`**: The extra parameters for Image WebSocket Web Viewer, see [websocket_nodes_extra_params.md](./websocket_nodes_extra_params.md)
- **URL Input:**
- **`url`**: This field is automatically updated with a constructed URL based on your inputs (server, channel, extra parameters, etc.). You can control its visibility with the **`show_url`** option.
3. **Open Web Viewer:**
- Click the **"Open Web Viewer"** button to launch the generated URL in a new browser window, where your image will be displayed in real time via the WebSocket connection.
4. **Outputs:**
- **`IMAGES`**: The input images are passed through as output.
- **`URL`**: The constructed URL for the web viewer.
**Notes:**
- Make sure that the server address and configuration are correct and that the server is accessible.
- This node uses the WebSocket protocol to transmit image data in real time to the specified channel; ensure your client browser supports WebSocket connections.
- When **`save_settings`** is enabled, a JSON with your websocket parameters is sent via the same WebSocket connection.
- When debug mode is enabled, the node outputs detailed logs to the console, which can help you track the image transmission process and troubleshoot any issues.
---
### Node: `IMAGE WebSocket Web Viewer @ vrch.ai` (vrch.ai/viewer)
A simplified version of the main WebSocket image viewer node that focuses purely on image transmission without settings management.
1. **Add the `IMAGE WebSocket Web Viewer @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Image Input:**
- **`images`**: Connect the image(s) you wish to display in the WebSocket-based web viewer.
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to differentiate WebSocket connections.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Image Format:**
- **`format`**: Choose the image format for transmission, supporting **PNG** and **JPEG** (default is **JPEG**).
- **Image Settings:**
- **`number_of_images`**: Set the number of images to load (default is **1**, range: 1-99).
- **`image_display_duration`**: Duration to display each image in milliseconds (default is **1000**, range: 1-10000).
- **`fade_anim_duration`**: Duration of fade animation in milliseconds (default is **200**, range: 1-10000).
- **Window Dimensions:**
- **`window_width`**: Set the width of the web viewer window (default is **1280**, range: 100-10240).
- **`window_height`**: Set the height of the web viewer window (default is **960**, range: 100-10240).
- **Show URL:**
- **`show_url`**: Toggle the display of the constructed URL in the interface. When enabled, the **`url`** field becomes visible (default is **False**).
- **Development Mode:**
- **`dev_mode`**: Enable development mode for additional features (default is **False**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting (default is **False**).
- **Extra Parameters:**
- **`extra_params`**: Additional parameters for the web viewer in string format (multiline text field).
- **URL Input:**
- **`url`**: This field is automatically updated with a constructed URL based on your inputs (server, channel, extra parameters, etc.). You can control its visibility with the **`show_url`** option.
3. **Open Web Viewer:**
- Click the **"Open Web Viewer"** button to launch the generated URL in a new browser window, where your image will be displayed in real time via the WebSocket connection.
4. **Outputs:**
- **`IMAGES`**: The input images are passed through as output.
- **`URL`**: The constructed URL for the web viewer.
**Notes:**
- This simplified node focuses purely on image transmission and does not include advanced WebSocket parameters or settings management.
- For workflows requiring custom WebSocket settings, use the **`IMAGE WebSocket Settings @ vrch.ai`** node in combination with this node.
- Make sure that the server address and configuration are correct and that the server is accessible.
- The `extra_params` field allows for additional customization of the web viewer behavior.
---
### Node: `IMAGE WebSocket Settings @ vrch.ai` (vrch.ai/viewer)
A dedicated node for managing and transmitting WebSocket settings parameters separately from image data. Supports optional merging of image CSS filter parameters supplied by the `IMAGE Filter Settings @ vrch.ai` node.
1. **Add the `IMAGE WebSocket Settings @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to differentiate WebSocket connections.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Send Settings Control:**
- **`send_settings`**: Toggle whether to actually send the settings to the WebSocket channel (default is **True**). When disabled, the node will skip settings transmission entirely, allowing you to prepare settings without broadcasting them.
- **Websocket Parameters:**
- **`number_of_images`**: Set the number of images to load (default is **1**, range: 1-99).
- **`image_display_duration`**: Duration to display each image in milliseconds (default is **1000**, range: 1-10000).
- **`fade_anim_duration`**: Duration of fade animation in milliseconds (default is **200**, range: 1-10000).
- **`blend_mode`**: Set the blend mode for image transitions (default is **"none"**). Options include **"normal"**, **"multiply"**, **"screen"**, **"overlay"**, **"darken"**, **"lighten"**, **"color-dodge"**, **"color-burn"**, **"hard-light"**, **"soft-light"**, **"difference"**, **"exclusion"**, **"hue"**, **"saturation"**, **"color"**, **"luminosity"**.
- **`loop_playback`**: Toggle whether to loop playback of images (default is **True**).
- **`update_on_end`**: Toggle whether to update the image cache only at the end of playback (default is **False**).
- **Background Settings:**
- **`background_colour_hex`**: Set the background color in hexadecimal format (default is **"#222222"**).
- **Server Messages:**
- **`server_messages`**: Server messages to send to the web page viewer (default is empty string).
- **Incremental Update:**
- **`incremental_update`**: When enabled, only changed settings fields are sent (diff payload). When disabled, the full settings block is sent every time (default is **False**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information; in incremental mode it also reports skipped updates (default is **False**).
- **Optional Filters JSON:**
- **`filters_json`** *(optional input)*: Connect the output of `IMAGE Filter Settings @ vrch.ai` (or any JSON node providing a compatible structure). When present, its filter data is merged into the outgoing settings under `settings.filters`.
3. **Settings Transmission:**
- When **`send_settings`** is enabled (default), this node saves the WebSocket settings to a JSON format and sends them via the WebSocket connection.
- When **`send_settings`** is disabled, the node will skip transmission and output a debug message (if debug mode is enabled) indicating that settings sending is disabled.
- Settings are transmitted to the specified channel and can be received by web viewers or other WebSocket clients.
- The settings JSON includes: numberOfImages, imageDisplayDuration, fadeAnimDuration, mixBlendMode, enableLoop, enableUpdateOnEnd, bgColourPicker, serverMessages, and (optionally) filters.
- With **`incremental_update`** enabled, the node remembers the previous settings per channel/server and only sends the fields that changed. If nothing changed, no JSON is produced.
4. **Outputs:**
- **`IMAGE_SETTINGS_JSON`**: The settings dictionary (Python/JSON) actually sent over WebSocket (only produced when `send_settings=True`). When incrementals are enabled and no values changed, the output is `None`. Structure example:
```json
{
"settings": {
"numberOfImages": 1,
"imageDisplayDuration": 1000,
"fadeAnimDuration": 200,
"mixBlendMode": "none",
"enableLoop": true,
"enableUpdateOnEnd": false,
"bgColourPicker": "#222222",
"serverMessages": "",
"filters": {
"opacity": 1.0,
"brightness": 1.2,
"contrast": 1.1,
"saturate": 1.4,
"hueRotate": 180,
"invert": 0.0,
"sepia": 0.2,
"blur": 4
}
}
}
```
**Notes:**
- Chain the `IMAGE Filter Settings @ vrch.ai` node into this node's `filters_json` input to send CSS filter parameters together with viewer settings.
- If `filters_json` contains a top-level `filters` key it will use that; otherwise the entire object is treated as the filters map.
- Output is only produced when settings are actually sent (`send_settings=True`).
- Use a JSON inspection node (or subsequent logic nodes) with the `IMAGE_SETTINGS_JSON` output for dynamic downstream control.
- Blend mode parameter provides extensive compositing options in the viewer.
- Filters are optional; if omitted, the viewer retains its current filter state or defaults (depending on the front‑end logic).
---
### Node: `IMAGE Filter Settings @ vrch.ai` (vrch.ai/viewer/websocket)
Provides adjustable CSS image filter parameters as a JSON object for composition into WebSocket image settings. Designed to feed into the `IMAGE WebSocket Settings @ vrch.ai` node via its `filters_json` input.
1. **Add the `IMAGE Filter Settings @ vrch.ai` node to your workflow.**
2. **Configure Filter Parameters (inputs):**
Order (updated): `opacity`, `brightness`, `contrast`, `saturate`, `hue_rotate`, `invert`, `sepia`, `blur`.
- **`opacity`**: Float (0.0–1.0) – 1.0 fully opaque.
- **`brightness`**: Float (0.0–2.0) – 1.0 neutral.
- **`contrast`**: Float (0.0–2.0) – 1.0 neutral.
- **`saturate`**: Float (0.0–2.0) – 1.0 neutral.
- **`hue_rotate`**: Integer degrees (0–360) – CSS `hue-rotate()`.
- **`invert`**: Float (0.0–1.0) – 0 none, 1 full invert.
- **`sepia`**: Float (0.0–1.0) – 0 none, 1 full sepia.
- **`blur`**: Integer pixels (0–50) – `blur(px)`.
(Deprecated: `grayscale` has been removed.)
3. **Advanced Options:**
- **`incremental_update`**: When enabled, only filter values that changed since the previous execution are emitted; otherwise a full filter block is produced each run.
- **`debug`**: Enable console logging for the node; in incremental mode it reports the diff or whether no changes were detected.
4. **UI Convenience:**
- A small green "Reset Filters" button (in the node UI) resets all parameters to their defaults.
5. **Output:**
- **`IMAGE_FILTERS_JSON`**: JSON structure (or `None` when incremental mode finds no differences):
```json
{
"filters": {
"opacity": 1.0,
"brightness": 1.0,
"contrast": 1.0,
"saturate": 1.0,
"hueRotate": 0,
"invert": 0.0,
"sepia": 0.0,
"blur": 0
}
}
```
5. **Typical Workflow:**
- Connect `IMAGE_FILTERS_JSON` to the `filters_json` input of the `IMAGE WebSocket Settings @ vrch.ai` node.
- Execute the settings node (with `send_settings=True`) to broadcast both regular settings and filters to the viewer.
**Notes:**
- The key names (`opacity`, `brightness`, `contrast`, `saturate`, `hueRotate`, `invert`, `sepia`, `blur`) match the front‑end's `applyRemoteFilters()` expectations.
- Legacy workflows containing `grayscale` will ignore that value silently.
- Values are clamped on the viewer side as a safety measure; staying within documented ranges ensures predictable results.
- You can route this JSON through additional logic or merge nodes before feeding into the settings node to implement animations or dynamic modulation.
---
### Node: `WebSocket Server @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `WebSocket Server @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Server Host:**
- **`server`**: Choose from `127.0.0.1` / `0.0.0.0` / your resolved default IP (from `VrchNodeUtils.get_default_ip_address(...)`).
- **Port:**
- **`port`**: TCP port for the server (default: **8001**).
- **External Server Only:**
- **`external_server_only`** *(optional, default **False**)*: Force external-proxy mode on the selected `server:port`.
- When **False** (default), behavior is unchanged (`auto` mode): if port is free, node can create/use built-in server.
- When **True**, node uses external-only mode: it does not create a new built-in server on that host:port and proxies to an existing external websocket service.
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
3. **Server Status & Full Address:**
- The node displays a status indicator that shows whether the server is running:
- **Grey "Stopped"**: The WebSocket server is not running
- **Green "Running"**: The WebSocket server is active and accepting connections
- Under the status, a centered full address label shows `HOST:PORT` and can be clicked to copy to clipboard. A small "copied" indicator briefly appears near the top‑right when the copy succeeds.
4. **Usage:**
- This node must be executed in your workflow to start the WebSocket server
- Once running, it handles communication for all WebSocket nodes (Image, JSON, Latent, Audio, Video, Text, and MIDI)
- The server maintains separate connection paths for different data types (/image, /json, /latent, /audio, /video, /text, /midi)
- Multiple clients can connect simultaneously to the same server
**Notes:**
- This node is required for any WebSocket-based communication in your workflow.
- Only one server can run on a specific IP:port combination.
- If a server is already running on the specified address and port, the node will use the existing server.
- With `external_server_only=True`, the node forces proxy behavior for the target address/port and is intended to pair with a standalone websocket service (for example `vrch-websocket-server.service` on `127.0.0.1:8001`).
- If `external_server_only=True` is enabled after a built-in server was already created in this process for the same host:port, the node will switch from built-in to proxy mode.
- WebSocket connections are maintained even when your workflow is not actively running.
- When debug mode is enabled, the server outputs detailed connection logs to the console.
---
### Node: `IMAGE WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `IMAGE WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to specify which WebSocket channel to listen on.
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **`placeholder`**: Choose the placeholder to display when no image data is received. Options:
- **"black"**: pure black placeholder image.
- **"white"**: pure white placeholder image.
- **"grey"**: mid-grey placeholder image.
- **"image"**: use the provided **`default_image`** as placeholder until a WebSocket image is available. Requires supplying **`default_image`**.
- **`default_image`**: *(Optional)* Image to use when **`placeholder`** is set to **"image"**.
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
3. **Outputs:**
- **`IMAGE`**: The output image tensor or placeholder.
- **`IS_DEFAULT_IMAGE`**: Boolean flag, `True` if the `IMAGE` output is the provided `default_image`, `False` otherwise.
4. **Receiving Images:**
- This node automatically connects to the specified WebSocket channel and listens for incoming image data.
- When an image is received, it will be processed and made available as the `IMAGE` output with `IS_DEFAULT_IMAGE` set to `False`.
- Once a WebSocket image has been received, the loader keeps returning the latest received image until a newer one arrives.
- If **`placeholder`** is set to **"image"** and no WebSocket image has been received yet, the **`default_image`** is used as the placeholder output with `IS_DEFAULT_IMAGE=True`.
**Notes:**
- This node is designed to work with the `IMAGE WebSocket Web Viewer @ vrch.ai` node, receiving the images it broadcasts.
- The node automatically establishes and maintains WebSocket connections, reconnecting if the connection is lost.
- The node continuously monitors for new images, allowing your workflow to react to images sent from any source that connects to the same WebSocket channel.
- When debug mode is enabled, the node outputs detailed logs to the console, which can help you track the image reception process and troubleshoot any issues.
---
### Node: `AUDIO WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
Receives audio streams sent over the WebSocket `/audio` path, decodes them to the ComfyUI audio tensor format, and provides graceful fallbacks for silence or custom default clips.
1. **Add the `AUDIO WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default **"1"**) to choose which audio stream to subscribe to.
- **`server`**: Enter the WebSocket server in `IP:PORT` format (defaults to **`127.0.0.1:8001`**).
- **`debug`**: Toggle verbose logging for connection status and decode errors (default **False**).
- **`default_audio`** *(optional)*: Provide an `AUDIO` tensor to use when no live audio has been received yet or decoding fails. Useful for pre-roll ambience or voice prompts.
3. **Outputs:**
- **`AUDIO`**: A dictionary with keys `waveform` (shape `[1, channels, samples]`) and `sample_rate`. When no stream is available and no default clip is supplied, the node emits a 0.5-second stereo silent buffer at 44.1 kHz.
4. **Receiving Audio:**
- The node maintains a persistent WebSocket subscription to `/audio` for the selected channel and automatically reconnects when needed.
- Incoming payloads are expected to contain base64-encoded **WebM** audio. The node pipes the data through `ffmpeg` and `torchaudio` to produce a normalized waveform tensor, auto-expanding mono inputs to stereo.
- If decoding fails or no payload has been received yet, the node returns the provided `default_audio`; otherwise it falls back to the generated silent clip.
**Notes:**
- Requires `ffmpeg` to be available in the environment (CLI executable or library). Missing `ffmpeg` will result in decode failures logged when `debug=True`.
- The output format matches the expectations of downstream VRCH audio visualization nodes (`waveform` batched, `sample_rate` integer).
- Pair with `AUDIO WebSocket Sender @ vrch.ai` (or other compatible publishers) and ensure the `WebSocket Server @ vrch.ai` node is running on the same host/port.
---
### Node: `AUDIO WebSocket Sender @ vrch.ai` (vrch.ai/viewer/websocket)
Sends a ComfyUI `AUDIO` tensor to Audio WebSocket Player over the shared `/audio` path. The player receives the clip, stores it in the browser playlist cache, and may play it when both sides allow autoplay.
1. **Add the `AUDIO WebSocket Sender @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **`audio`**: Connect the ComfyUI audio clip to send.
- **`channel`**: Select a channel number from **"1"** to **"8"** (default **"1"**). This appears before `server` so channel selection stays the primary routing choice.
- **`server`**: Enter the WebSocket server in `IP:PORT` format (defaults to **`127.0.0.1:8001`**).
- **`title`**: Playlist row name shown in Audio WebSocket Player (default **`ComfyUI Audio`**). The filename is derived from this title and a message id.
- **`autoplay_request`**: Request immediate playback after the player caches the clip (default **True**). Audio Player only honors this when its local **Auto Play** toggle is also enabled.
- **`quality`**: Opus bitrate preset for WebSocket transport:
- **`compact`**: 64 kbps
- **`standard`**: 128 kbps (default)
- **`high`**: 192 kbps
- **`debug`**: Print concise send metadata without base64 payloads (default **False**).
3. **Outputs:**
- **`AUDIO`**: Pass-through original audio.
- **`PAYLOAD`**: Metadata-only JSON payload for inspection. The full base64 audio body is omitted from this output.
4. **Payload Contract:**
- The sender emits a typed JSON envelope with root `type="vrch_audio_player_track"` and `target="audio_player_playlist"`.
- The sender does **not** put `base64_data` at the root. Root `base64_data` remains reserved for legacy Browser -> ComfyUI audio input consumed by `AUDIO WebSocket Channel Loader @ vrch.ai`.
- The audio body is WebM/Opus with MIME type `audio/webm`, base64-encoded under `audio.base64`.
- Audio Player ignores this payload unless `Receive WebSocket Audio` is enabled.
**Notes:**
- Requires `ffmpeg` with WebM muxing and Opus encoding support (`libopus` or native `opus`).
- WebM/Opus is used as the transport format to avoid the bandwidth overhead of raw WAV.
- If Audio Player has both **Auto Play** and **Send audio chunks via /audio** enabled, an autoplayed generated clip can be sent back to ComfyUI as legacy audio chunks during playback. Disable either local option if that echo-back is not desired.
---
### Node: `JSON WebSocket Sender @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `JSON WebSocket Sender @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **JSON Input:**
- **`json_string`**: Enter the JSON string you want to send over WebSocket. This should be a properly formatted JSON string.
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to differentiate WebSocket connections.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
3. **Sending JSON Data:**
- The node validates the JSON string before sending. If the string is not valid JSON, the node will raise a ValueError.
- Once validated, the JSON data is sent to the specified WebSocket channel and path (/json).
- The validated JSON is also available as an output that can be connected to other nodes.
**Notes:**
- This node works with the `WebSocket Server @ vrch.ai` node, which must be running to establish connections.
- Ensure your JSON string is properly formatted to avoid validation errors.
- The JSON data is sent through the WebSocket as a raw string.
- When debug mode is enabled, the node outputs detailed logs to the console, including the content of the sent JSON data.
---
### Node: `LIVE Console Control @ vrch.ai` (vrch.ai/viewer/websocket)
This node is designed for controlling `vrch_live_console.html` panes from a ComfyUI workflow through `/json`.
1. **Add the `LIVE Console Control @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Connection:**
- **`server`**: WebSocket server in `IP:PORT` format (for example `127.0.0.1:8001`).
- **`channel`**: Channel **"1"** to **"8"** used by Live Console (default **"8"**).
- **Behavior:**
- **`collapse_sidebar`**: Collapse the Live Console sidebar when enabled (default **`True`**).
- **`only_send_changed`**: When enabled, only pane/sidebar states changed since last execution are sent.
- **`debug`**: Enable verbose logs.
- **Pane Visibility Toggles (boolean):**
- `display_image_viewer`
- `display_image_viewer_aux`
- `display_image_settings`
- `display_shader_editor`
- `display_prompt_sender`
- `display_srt_player`
- `display_sketch_sender`
- `display_image_sender`
- `display_audio_recorder`
- `display_audio_player`
- `display_midi_sender`
- `display_gamepad_sender`
- Each toggle directly represents target visibility (`true` = show, `false` = hide).
3. **Sending Control Data:**
- The node sends a `/json` payload with a dedicated top-level key:
```json
{
"live_console_control": {
"version": "1.0",
"request_id": "lc-1760000000000",
"timestamp_ms": 1760000000000,
"ops": [
{
"op": "sidebar.set_mode",
"target": "sidebar",
"args": { "mode": "thin" }
},
{
"op": "pane.set_visibility",
"target": "audioPlayer",
"args": { "visible": true }
}
],
"meta": {
"source": "VrchLiveConsoleControlNode"
}
}
}
```
4. **Live Console Execution Rules (receiver side):**
- Receiver reads `live_console_control.ops[]`.
- Supported V1 operations:
- `sidebar.set_mode`
- `pane.set_visibility`
- Unknown operations or unknown targets are safely ignored.
- Control channel handling should be independent from status-bar visibility (status bar on/off must not affect control).
5. **Notes:**
- V1 intentionally uses standard Comfy widgets only (no customWidget required).
- The `ops[]` envelope is reserved for future extensibility (layout control, splitter lock, section focus, etc.) without breaking compatibility.
- `show/hide` semantics are idempotent and preferred for automation stability.
---
### Node: `JSON WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `JSON WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to specify which WebSocket channel to listen on.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
- **Default JSON:**
- **`default_json_string`**: (Optional) Enter a default JSON string to use when no data is received. This should be a properly formatted JSON string.
3. **Receiving JSON Data:**
- This node automatically connects to the specified WebSocket channel and listens for incoming JSON data.
- When JSON data is received, it will be parsed and made available as an output that can be connected to other nodes in your workflow.
- Incoming JSON **dict** payloads are shallow‑merged by top‑level keys into a persistent state for that channel. Non‑dict JSON payloads are ignored.
- The merged state is capped at **128 top‑level keys**; new keys beyond the limit are ignored.
- Send `{"__clear__": true}` to clear the merged state (you may include additional keys in the same message after clearing).
- If no JSON data has been received yet, the **`default_json_string`** is used (if provided). Otherwise an empty object `{}` is returned.
- If the provided default JSON string is not valid JSON, the node will raise a ValueError.
**Notes:**
- This node is designed to work with the `JSON WebSocket Sender @ vrch.ai` node, receiving the JSON data it broadcasts.
- The node automatically establishes and maintains WebSocket connections, reconnecting if the connection is lost.
- The node continuously monitors for new JSON data, allowing your workflow to react to data sent from any source that connects to the same WebSocket channel.
- When debug mode is enabled, the node outputs detailed logs to the console, which can help you track the JSON data reception process.
---
### Node: `MIDI WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
Receives optimized binary MIDI control state from `vrch_midi_websocket_sender.html` over `/midi?channel=N`.
1. **Add the `MIDI WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**).
- **`server`**: Enter the WebSocket server in `IP:PORT` format. The default uses your resolved host and port **8001**.
- **`debug`**: Enable parser timing and lookup diagnostics.
3. **Output:**
- **`MIDI`**: A `VRCH_MIDI` state object consumed by MIDI control nodes. See [MIDI Control Nodes](./midi_control_nodes.md).
4. **Protocol:**
- The sender uses binary `VMID` frames.
- Definition frames carry metadata: `workflow_key`, label, sender id, binding type, MIDI channel, and CC/note number.
- State frames carry compact raw CC values, note values, and control-index values.
- Preset sync is not handled by this loader; it remains on `/json` between MIDI Sender pages.
5. **Lookup Example:**
If MIDI Sender shows a control:
```text
Label: Brightness
Workflow Key: brightness
Binding: CC 22
```
Use:
```text
lookup_mode = workflow_key
control_key = brightness
```
In this mode `midi_channel` and `cc_number` are ignored. `Brightness` is only a display label and is not a receiver lookup key.
**Notes:**
- The loader returns a valid empty `VRCH_MIDI` state before data arrives.
- Use the Workflow Key shown in MIDI Sender for non-technical workflows. Direct CC lookup is available when you need physical MIDI routing.
- Definition frames are resent periodically by the sender, so late-joining loaders can recover metadata.
---
### Node: `LATENT WebSocket Sender @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `LATENT WebSocket Sender @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Latent Input:**
- **`latent`**: Connect the latent data you want to send over WebSocket. This should be a properly formatted latent tensor.
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to differentiate WebSocket connections.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
3. **Sending Latent Data:**
- The node validates the latent data before sending. If the latent data is invalid, the node will raise a ValueError.
- The latent tensor is converted to JSON format (including shape information) and sent to the specified WebSocket channel and path (/latent).
- The original latent data is also available as an output that can be connected to other nodes.
**Notes:**
- This node works with the `WebSocket Server @ vrch.ai` node, which must be running to establish connections.
- Ensure your latent data contains valid samples to avoid validation errors.
- The latent data is converted to JSON format for transmission and includes shape information for proper reconstruction.
- When debug mode is enabled, the node outputs detailed logs to the console, including the shape of the sent latent data.
---
### Node: `LATENT WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `LATENT WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to specify which WebSocket channel to listen on.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Latent Format:**
- **`latent_format`**: Choose the expected latent format. **`SD1/SDXL`** uses 4 channels; **`SD3/FLUX`** uses 16 channels.
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
- **Default Latent:**
- **`default_latent`**: (Optional) Latent data to use when no data is received from the WebSocket channel.
3. **Receiving Latent Data:**
- This node automatically connects to the specified WebSocket channel and listens for incoming latent data.
- When latent data is received, it will be parsed and reconstructed into the proper tensor format for use in your workflow.
- If the received latent channels do not match the selected **`latent_format`**, the node outputs an empty latent (or the **`default_latent`** if it matches the selected format).
- If no latent data has been received yet, the default latent will be used (if provided), otherwise an empty latent tensor is created using the selected format.
- The empty latent has dimensions (1, 4, 64, 64) for **SD1/SDXL** or (1, 16, 64, 64) for **SD3/FLUX**.
**Notes:**
- This node is designed to work with the `LATENT WebSocket Sender @ vrch.ai` node, receiving the latent data it broadcasts.
- The node automatically establishes and maintains WebSocket connections, reconnecting if the connection is lost.
- The node continuously monitors for new latent data, allowing your workflow to react to latent data sent from any source that connects to the same WebSocket channel.
- When debug mode is enabled, the node outputs detailed logs to the console, which can help you track the latent data reception process and show tensor shapes.
---
### Node: `JSON WebSocket Channel Loader @ vrch.ai` (vrch.ai/viewer/websocket)
1. **Add the `JSON WebSocket Channel Loader @ vrch.ai` node to your ComfyUI workflow.**
2. **Configure the Node:**
- **Channel:**
- **`channel`**: Select a channel number from **"1"** to **"8"** (default is **"1"**) to specify which WebSocket channel to listen on.
- **Server:**
- **`server`**: Enter the server's domain or IP address along with its port in the format `IP:PORT`. The default typically uses your IP and port **8001** (e.g., **`127.0.0.1:8001`**).
- **Debug Mode:**
- **`debug`**: Enable this option to print detailed debug information to the console for troubleshooting.
- **Default JSON:**
- **`default_json_string`**: (Optional) Enter a default JSON string to use when no data is received. This should be a properly formatted JSON string.
3. **Receiving JSON Data:**
- This node automatically connects to the specified WebSocket channel and listens for incoming JSON data.
- When JSON data is received, it will be parsed and made available as an output that can be connected to other nodes in your workflow.
- Incoming JSON **dict** payloads are shallow‑merged by top‑level keys into a persistent state for that channel. Non‑dict JSON payloads are ignored.
- The merged state is capped at **128 top‑level keys**; new keys beyond the limit are ignored.
- Send `{"__clear__": true}` to clear the merged state (you may include additional keys in the same message after clearing).
- If no JSON data has been received yet, the **`default_json_string`** is used (if provided). Otherwise an empty object `{}` is returned.
- If the provided default JSON string is not valid JSON, the node will raise a ValueError.
**Notes:**
- This node is designed to work with the `JSON WebSocket Sender @ vrch.ai` node, receiving the JSON data it broadcasts.
- The node automatically establishes and maintains WebSocket connections, reconnecting if the connection is lost.
- The node continuously monitors for new JSON data, allowing your workflow to react to data sent from any source that connects to the same WebSocket channel.
- When debug mode is enabled, the node outputs detailed logs to the console, which can help you track the JSON data reception process.
Reference in New Issue View Git Blame Copy Permalink