Jedrzej KosinskiandAmp
04ca70a853
The proxy_node has accepted VIDEO outputs since PR #13 (server-side `make_video_envelope` + client-side `decode_video_envelope`), but the inbound encoding direction has been a no-op: `_encode_one` in `proxy_node.py` only dispatched AUDIO + IMAGE + MASK tensors, so any remote node declaring an `IO.Video.Input` would receive the raw `VideoInput` object as a non-JSON-serialisable Python value and the upstream serialisation would either drop it or fail. `serialization.py:468` made the gap explicit with a placeholder comment ("encode lands when a remote node accepts VIDEO inputs"), and several server-side providers shipped against the `VIDEO_MP4_BASE64` capability + `input_serialization={"video": "mp4_base64"}` declarations on the assumption that the client encoder would land soon — most notably `GrokVideoEditProvider` and `GrokVideoExtendProvider`, whose end-to-end VIDEO upload path was broken until now. This change wires the encode direction: * `serialization.py` gains `encode_video_input(video)` — mirrors the AUDIO encoder above. Re-encodes the `VideoInput` to an in-memory mp4/H.264 byte buffer via the upstream `comfy_api_nodes/util/conversions.video_to_base64_string` (same call shape: `video.save_to(buf, format=MP4, codec=H264)`) and base64-encodes the result. Populates `duration_s` from `video.get_duration()` so server-side providers (e.g. `_validate_video_duration_envelope` in `grok.py`) can range-check without demuxing the MP4. * `serialization.py` gains `is_video_input(value)` — duck-types on the `save_to` + `get_duration` method pair (both defined on `comfy_api.latest._input.VideoInput` and present on every concrete subclass). Avoids importing `VideoInput` at module load (which would pull torch via the `comfy_api` tree). Both helpers added to `__all__`. * `proxy_node.py:_encode_one` gains a VIDEO branch right after the AUDIO branch (and before the tensor-rank dispatch), mirroring the AUDIO pattern at the same call site. * `proxy_node.py` `_inputs_to_envelopes` docstring updates the duck-typing-rules list to mention the new VIDEO branch. Symmetric to the (still-open) MODEL_3D INPUT gap noted at `serialization.py:484`. AUDIO INPUT was already wired (ElevenLabs nodes exercise it end-to-end); IMAGE / MASK have always been wired; VIDEO INPUT closes today's last input-direction envelope gap for the existing capability vocabulary. Verified by importing `serialization` with a stubbed `comfy_remote_nodes` package: `is_video_input` returns False for None / dict / audio-shaped dict and True for an object exposing both `save_to` + `get_duration`. `encode_video_input` + the proxy_node integration are exercised end-to-end by the server-side `Wan2VideoContinuationApi_RNP` provider landing in the matching comfy-rnp-server PR (it consumes `first_clip: VIDEO` and uploads the resolved bytes to Comfy storage), and also unblocks the shipped-but-broken Grok VideoEdit/VideoExtend providers. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e35d1-3c68-74ce-9918-fe9cacd74276
comfy-remote-nodes
ComfyUI custom-node package that fetches node descriptors from an RNP/1 (Remote Node Protocol v1) server and exposes them as native V3 nodes inside ComfyUI.
The reference server lives at
Comfy-Org/comfy-rnp-server
(private). This client is intentionally provider-agnostic: any server
that speaks RNP/1 can publish nodes through it.
Install
Clone into your ComfyUI custom_nodes/ directory:
cd ComfyUI/custom_nodes
git clone https://github.com/Comfy-Org/comfy-remote-nodes.git
(The directory name on disk doesn't matter — ComfyUI loads the package
via spec_from_file_location, so comfy-remote-nodes/ is fine.)
Configure
Point the extension at one RNP server URL via env var before starting ComfyUI:
export RNP_SERVER_URL=http://127.0.0.1:9190
python main.py
If RNP_SERVER_URL is unset, the extension registers zero nodes and
logs a single line — ComfyUI starts normally with no remote nodes.
What works today
- Manifest + capability handshake
object_infofetch and dynamic V3 class generation (with ETag conditional GETs)- Sync
executeand asyncexecute_async+ poll/cancel - Image / mask / audio / video envelope encode + decode
- Presigned upload helper for >8 MiB inputs
- Tier 3 (recent):
TASK_LOSTdetection with idempotent resubmission — survives an RNP server restart mid-task without losing the workflowSERVER_BUSY/MAINTENANCE503 envelopes treated as rate limits withRetry-Afterhonoring and friendly per-second progress text ("Server busy, retrying in 30s...")- Out-of-band
url_fetchfor URI-only output envelopes, driven by the descriptor'sremote.url_fetchpolicy - Per-poll auth refresh so long-running tasks can rotate tokens
What does not (yet)
validate_inputs/fingerprint_inputs/check_lazy_status— the server returns501 NOT_IMPLEMENTED; the client falls through to ComfyUI's defaults- Multi-server config file (
comfy_remote_nodes.yamlor similar) — today's setup is one server per ComfyUI process via env var - Streaming execute (server-sent events for incremental previews)