feat/task-handle-decoder
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jedrzej KosinskiandAmp
|
1f136526cf |
Add client-side cross-node task_handle decoder + TaskHandle dataclass
Mirrors comfy-rnp-server PR #39 on the client. Adds the surfaces needed for chained partner provider nodes (Tripo Texture/Refine/ Rig/Retarget/Conversion, Kling VideoExtend, future partner chained nodes) to pass vendor-native task / video / job IDs through a workflow with their lineage metadata intact. Surfaces: * ``protocol.Capability.IO_TASK_HANDLE = "io:task_handle"`` and ``"task_handle"`` added to ``protocol.HEAVY_TYPES`` so ``is_envelope`` recognises the new envelope. * ``serialization.TaskHandle`` dataclass with the wire-shape fields (``vendor``, ``kind``, ``native_id``, ``origin_node_id``, ``parent_chain``). ``__str__`` surfaces ``native_id`` for log / UI preview without leaking the lineage chain. * ``serialization.decode_task_handle_envelope`` validates every field (mirroring server-side ``_validate_task_handle_ref`` so a bad envelope surfaces the same error message on both sides) and returns the dataclass. Rejects: - wrong encoding (only ``vendor_inline`` recognised); - missing / empty top-level fields; - comma-union ``kind`` strings (the input-acceptance string used by some provider nodes, NOT an emitted handle kind); - non-list ``parent_chain``; - malformed parent_chain entries (missing fields, non-dict, comma-union kind in lineage). * ``serialization.encode_task_handle`` re-encodes the dataclass back to wire shape — symmetric counterpart of the decoder so a handle decoded from one provider's output and fed into another provider's input round-trips losslessly without ``proxy_node`` having to know anything provider-specific. * ``decode_envelope`` dispatcher routes ``type="task_handle"`` to the new decoder so the generic ``_deserialize_output`` path picks it up. * ``client.CLIENT_CAPABILITIES`` advertises ``Capability.IO_TASK_HANDLE`` so server-side capability-gated descriptors (whole-descriptor gating — partial-socket synthesis is rejected because a chain that breaks meaningfully if either end is missing is worse than no chain at all) negotiate cleanly. The dataclass return (NOT a plain ``str``) is deliberate: a plain string would silently drop ``parent_chain`` when the value is passed through the workflow, breaking the lineage the server uses for future cascading-replay semantics (deferred to a follow-up PR alongside server-side replay-cache storage — see PR #39 for the replay design notes). Smoke test ``notes/run_task_handle_decoder.py`` mirrors the pattern used by ``run_model3d_bundle_decoder.py``: stubs out the ``client`` import for the relative ``from . import client as rnp_client`` at the top of serialization.py, then exercises the 9 contract points end-to-end. End-to-end check imports the server's ``make_task_handle_envelope`` from the comfy-rnp-server worktree and verifies the wire shape round-trips losslessly through decode -> encode. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e3431-c416-742b-973c-e262ab13f2ff |
||
|
Jedrzej KosinskiandAmp
|
c962fa5adf |
Add client-side multi-file 3D bundle decoder + BundledFile3D
Closes the client side of the multi-file 3D bundle envelope landed in
comfy-rnp-server#38 (commit 50f9f4f on main).
* Vendored `protocol.py`: mirrors the new
`Capability.MODEL_3D_BUNDLE_INLINE = "model_3d:bundle_inline"`
constant with a full docstring describing the wire shape, path /
role semantics, primary_path / format invariants, capability
negotiation discipline, and the planned future
`MODEL_3D_BUNDLE_URI` sibling. HEAVY_TYPES is unchanged
(`type="model_3d"` already covers the new encoding).
* `serialization.py`: adds `BundledFile3D` as a `File3D` subclass
built lazily via a `_bundled_file3d_class()` factory (so importing
the heavy `comfy_api.latest._util` package doesn't happen at
module load — keeps the smoke-test stubbing pattern intact for
callers that don't exercise the bundle path). The class stores
the primary mesh file plus N companion files in memory as a
`path -> bytes` dict and:
- `get_data()` / `get_bytes()` return the primary file's bytes
without touching disk.
- `get_source()` lazily materialises the whole bundle into a temp
directory using the producer-supplied relative paths, then
returns the on-disk path to the primary file — so a .obj that
references its .mtl by filename (and the .mtl that references
`textures/albedo.png`) resolves via the filesystem.
- `save_to(dest)` copies the WHOLE bundle into the destination
directory: the primary file under the caller-specified name,
sidecars under their ORIGINAL relative paths so cross-file refs
inside the .obj that point at `model.mtl` still resolve when
the user saves the model somewhere stable.
- New convenience accessors `primary_path`, `file_paths`,
`file_bytes(path)`, `file_role(path)`, `paths_with_role(role)`,
and `materialize_to_temp_dir()`.
- Temp directory cleanup on garbage collect (best-effort
`shutil.rmtree(td, ignore_errors=True)` in `__del__`).
- `super().__init__(BytesIO(primary), primary_fmt)` keeps the
upstream `_source` / `_format` attributes populated so existing
downstream consumers that read those plain attributes still
work.
`decode_model3d_envelope` now dispatches on `encoding`:
`glb_inline` falls through to the existing single-file path
unchanged; `bundle_inline` walks the new wire shape, decodes
per-file base64 `data`, hoists `role` / `format` hints into
parallel dicts, and constructs a `BundledFile3D`. Malformed
envelopes (missing `primary_path`, missing `files`, primary not
present in files, file entry missing `data`) raise
`RnpProtocolError(code=INTERNAL)` rather than falling through to
any opaque bucket.
* `client.py`: advertises `Capability.MODEL_3D_BUNDLE_INLINE` in
`CLIENT_CAPABILITIES` with a full docstring matching the
established pattern for the other 3D capabilities.
Design decisions (from oracle consult on wire-shape surface):
* Extend the existing `model_3d` heavy type with a new encoding
(rather than introducing `model_3d_bundle` as a separate
top-level type) — keeps HEAVY_TYPES unchanged, lets the
single-file `glb_inline` decoder stay untouched, and lets
capability gating stay parallel.
* `path` is the authoritative cross-file reference key (not
`role`) — roles are conceptually non-unique (multiple meshes,
LODs, buffers, animations) so `primary_path` pins the entry
point unambiguously.
* `role` is an OPTIONAL producer hint, NOT a closed enum. Unknown
roles round-trip without error.
* Subclass `File3D` directly rather than introducing a new opaque
`ModelBundle3D` type — existing downstream loaders that
type-check or duck-type-check on `File3D` keep working.
`isinstance(bundled, File3D)` succeeds.
* Each file's `data` is base64 (mirrors the single-file
`glb_inline` payload mode). Mixed `data`/`uri` inside one
encoding was rejected per the parallel-encoding pattern already
in use across the rest of the protocol; a future sibling
`Capability.MODEL_3D_BUNDLE_URI` would handle URL-fetch
semantics.
* Capability gating with whole-descriptor enforcement — if a node
has both bundle and non-bundle outputs and the cap is absent,
the whole descriptor is skipped rather than synthesising a
partial-socket variant.
Smoke test at `notes/run_model3d_bundle_decoder.py` — 9 checks:
1. Capability constant intact at protocol layer.
2. is_envelope recognises bundle_inline envelopes.
3. decode dispatch returns BundledFile3D (a File3D subclass) +
primary bytes round-trip via get_data / get_bytes.
4. get_source materialises whole bundle on disk at producer-
supplied paths.
5. save_to copies whole bundle: primary renamed, sidecars keep
filenames.
6. Bundle accessors (primary_path / file_paths / file_role /
paths_with_role) work; unknown roles round-trip.
7. Malformed bundle envelopes raise RnpProtocolError(INTERNAL).
8. End-to-end against server's make_model3d_bundle_envelope helper
(from PR #38) — all files / roles / bytes preserve. Server-side
path sanitization rejects `..` at envelope build time
(defence-in-depth).
9. CLIENT_CAPABILITIES advertises model_3d:bundle_inline.
All 9 pass; PR #1's `notes/run_model3d_decoder.py` (8 checks) and
PR #2's `notes/run_dynamic_combo_parser.py` (8 checks) also still
pass — no regression on existing paths.
Hunyuan3D / Tencent / future Meshy / Rodin / Tripo bundle
providers land in a follow-up PR that consumes this decoder; this
PR is protocol + client decoder only per the planned sequence.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e331c-c202-746b-96ec-86ce0b9c54a5
|
||
|
Jedrzej KosinskiandAmp
|
58d93daf2a |
feat(model_3d): add client-side MODEL_3D decoder + capability advertisement
Closes the loop on the just-merged comfy-rnp-server PR #37 which added the four Tripo / Rodin 3D providers and the ``Capability.MODEL_3D_GLB_INLINE`` token. Without this client-side half the providers were unreachable from any RNP client — descriptors would fail negotiation because the proxy_node didn't advertise the token. Client changes (all inside ``ComfyUI/custom_nodes/comfy_remote_nodes``): * ``protocol.py`` — mirror the new ``Capability.MODEL_3D_GLB_INLINE`` constant and add ``"model_3d"`` to ``HEAVY_TYPES`` so ``is_envelope`` recognises inbound 3D-model envelopes. The vendored ``protocol`` module is the single source of truth for the proxy_node and serialization layers; both already consume ``is_envelope`` / ``decode_envelope_data`` from here. * ``serialization.py`` — add ``decode_model3d_envelope`` that hands raw GLB bytes to ``comfy_api.latest._util.File3D`` via ``BytesIO``, mirroring the existing ``decode_video_envelope`` shape. Only ``glb_inline`` is accepted today; any other encoding surfaces ``RnpProtocolError(code=INTERNAL)`` as defence-in-depth so a future encoding never silently downgrades to garbage bytes. Registered in the ``_DECODERS`` dispatch table and re-exported via ``__all__``. * ``proxy_node.py`` — add ``"MODEL_3D" -> IO.File3DGLB.Output`` to ``_OUTPUT_CLASSES`` so descriptors with ``output=["MODEL_3D"]`` parse to a real socket. The descriptor io_type string is the logical name ``"MODEL_3D"`` (consistent across vendors) — the resulting V3 Output carries the upstream ``"FILE_3D_GLB"`` comfytype tag so frontend-side socket matching works against any GLB-aware downstream node (preview3d / SaveGLB / Load3D). * ``client.py`` — advertise ``Capability.MODEL_3D_GLB_INLINE`` in ``CLIENT_CAPABILITIES`` so servers gate provider descriptors on the presence of the token. Legacy clients that don't advertise it still see NEGOTIATION_FAILED at descriptor-load time (safe degradation), not a parse error at execute. Smoke test (``notes/run_model3d_decoder.py``) covers: 1. ``HEAVY_TYPES`` includes ``"model_3d"``. 2. ``is_envelope`` recognises a MODEL_3D envelope. 3. ``decode_model3d_envelope`` round-trips a minimal valid GLB (binary glTF 2.0, magic ``b"glTF"``). 4. ``decode_envelope`` dispatches ``type=model_3d`` to the new decoder. 5. Unsupported encoding raises ``RnpProtocolError``. 6. End-to-end against the server's ``make_model3d_envelope`` helper — server packs, client decodes, bytes match. 7. ``proxy_node._OUTPUT_CLASSES`` contract: ``"MODEL_3D" -> IO.File3DGLB.Output``. 8. ``client.CLIENT_CAPABILITIES`` advertises ``Capability.MODEL_3D_GLB_INLINE``. No server-side changes. The 4 just-shipped Tripo / Rodin providers (``TripoTextToModelApi_RNP``, ``TripoImageToModelApi_RNP``, ``TripoMultiviewToModelApi_RNP``, ``Rodin3DRegularApi_RNP``) are now reachable end-to-end from ComfyUI. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e3314-4336-71d2-a333-6d1e33a9986e |
||
|
Jedrzej KosinskiandAmp
|
cd3e051956 |
feat(audio): add Capability.AUDIO_OPUS_BASE64 + decode_audio_envelope branch
Pairs with comfy-rnp-server feat/audio-opus-base64. Lets the proxy_node accept and decode `audio:opus_base64` envelopes (OGG/ Opus container as base64) so audio-out partner-node providers whose upstream returns OGG/Opus bytes (e.g. ElevenLabs `output_format=opus_48000_192`) can ship an honest envelope instead of label-laundering the bytes as `mp3_base64`. protocol.py: - `Capability.AUDIO_OPUS_BASE64 = "audio:opus_base64"`. client.py: - `CLIENT_CAPABILITIES` now includes `Capability.AUDIO_OPUS_BASE64`, matching the AUDIO sweep precedent (mp3 + wav already in the list). serialization.py: - `decode_audio_envelope` allow-list extended with `opus_base64`. PyAV/FFmpeg already decode OGG/Opus natively in `audio_bytes_to_audio_input`, so the decoder branch is a one-line allow-list change — same path as MP3/WAV, just a different container. The wire-format hint is now honest rather than label- laundering, since the inbound envelope's `encoding` matches the actual container. The capability negotiation is symmetric to the existing `image:png_base64_batch` / `audio:mp3_base64` negotiation: servers gate emission on the inbound advertise list, and the per-format fallback (mp3) is required while opus is opportunistic. Amp-Thread-ID: https://ampcode.com/threads/T-019e2e33-da42-74c2-a6cf-8c7942cb8180 Co-authored-by: Amp <amp@ampcode.com> |
||
|
KosinkadinkandAmp
|
6433a46c93 |
feat(serialization): externalize oversize image batch envelopes via per-frame presigned URIs
The pre-existing `serialization.maybe_externalize` only inspected the
singleton `data` field, so a multi-frame `frames=[…]` IMAGE envelope
whose summed base64 payload crossed the 8 MiB
`DEFAULT_MAX_INLINE_PAYLOAD_BYTES` cap silently rode the wire as a
30+ MiB POST body — defeating the entire reason the cap exists. A
9× 2048² Flux2 reference batch hits ~200 MB base64 inline; that's
the worst-case the new path is designed for.
This commit adds the **client side** of the lockstep fix: detect
oversize multi-frame batches (sum of per-frame raw bytes), upload
each frame as an individual presigned PUT in parallel via
`asyncio.gather`, and rewrite the envelope to the new
`image:png_base64_batch_uri` encoding with a parallel `uris` list.
Server side closes the loop with a matching encoding-string
dispatch in `resolve_image_envelope_pngs`.
Per-frame URIs (NOT a tar/zip blob) compose with the existing
per-frame fetch helpers and let the server fetch in parallel.
## Server
- See lockstep PR on `Comfy-Org/comfy-rnp-server` branch
`feat/batch-envelope-externalization` for the
`Capability.IMAGE_PNG_BASE64_BATCH_URI` token, the
`_resolve_png_base64_batch_uri` per-frame fetch helper, and the
encoding-string dispatch in `resolve_image_envelope_pngs`.
## Client
- `protocol.py`: mirror new `Capability.IMAGE_PNG_BASE64_BATCH_URI =
"image:png_base64_batch_uri"` token next to
`IMAGE_PNG_BASE64_BATCH`. Same comment block as the server-side
copy so the two stay byte-comparable.
- `client.py`: append `Capability.IMAGE_PNG_BASE64_BATCH_URI` to
`CLIENT_CAPABILITIES` so the server knows this client can emit
the externalised batch shape. Comment notes the opportunistic
posture (any input that already accepts `png_base64_batch`
transparently accepts the externalised variant — no descriptor
opt-in needed).
- `serialization.py`:
- `maybe_externalize` now branches on payload shape: if
`envelope["frames"]` is a populated list, dispatch to the new
`_maybe_externalize_batch` helper; otherwise fall through to
the existing singleton-`data` path. Docstring expanded to
describe both shapes and the encoding-flip-on-spill behaviour.
- New `_maybe_externalize_batch` helper sums per-frame raw bytes
(`len(b64) * 3 // 4`), short-circuits when the batch fits under
the cap, and otherwise uploads every frame in parallel via
`asyncio.gather` (preserves frame ordering — critical because
Flux2 maps `uris[k]` → `input_image_{k+1}` 1:1) before
rewriting the envelope to `{type, encoding:
png_base64_batch_uri, uris: [...], shape, dtype, color_space}`
with `frames` stripped.
- `import asyncio` added at module top so `asyncio.gather` is
available without per-call imports.
## Docs
- The pilot side-by-side workflow `notes/sxs_flux2_pro_multi_ref_xl.json`
(9× 2048² references) and the closed-gap entry in
`notes/remote-nodes-vocabulary-gaps.md` live in the server
worktree (this client repo has no `notes/` directory). Both ship
in the paired server PR.
## Verification
- AST parse of `protocol.py`, `serialization.py`, `client.py`
succeeds.
- In-process round-trip test (with a fake `comfy_api_nodes.util.client`
shim and a stubbed `upload_asset`):
* Under-cap batch envelope (1× 5 MiB raw, cap=8 MiB) → passes
through unchanged (object identity preserved).
* Over-cap batch envelope (3× 5 MiB raw, cap=8 MiB) → triggers
3 PUTs via the stub, returns
`{type: image, encoding: png_base64_batch_uri,
uris: ['…/asset/1','…/asset/2','…/asset/3'], shape: [3,1,1,3],
dtype: uint8, color_space: srgb}` with `frames` stripped.
* Singleton `data` envelope (10 MiB raw, cap=8 MiB) → still gets
the legacy `data → uri` swap (existing path unaffected).
* `Capability.IMAGE_PNG_BASE64_BATCH_URI in CLIENT_CAPABILITIES`
holds.
## Lockstep
- Paired server PR: https://github.com/Comfy-Org/comfy-rnp-server/pull/14
(`feat/batch-envelope-externalization`). Server PR was opened
first because the wire shape needs a server resolver to be
meaningful — the orchestrator merges them in lockstep order.
- Capability gating posture: `IMAGE_PNG_BASE64_BATCH_URI` is
opportunistic — this client emits it whenever a batch envelope it
produced overflows the inline cap, regardless of whether the
descriptor's `input_serialization` lists it. Server accepts on
any image input that already accepts `png_base64_batch` via the
shared `resolve_image_envelope_pngs` dispatch.
- Output-side decoder: server providers emit `png_base64_batch` (or
single-frame `png_base64`) on responses today; the URI shape is
consumed on input only, so the client decoder (`_DECODERS` /
`_decode_image_batch_envelope`) is unchanged. The existing
`decode_image_envelope` already dispatches on the `encoding`
string (not field presence), so adding a `png_base64_batch_uri`
decode branch is a drop-in if a future provider starts emitting
the URI shape on outputs. Logged as a known follow-up.
Amp-Thread-ID: T-019e29b4-d606-7278-8584-52910661c981
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e2adb-6ddf-72ad-88e1-8b30c57e873f
Co-authored-by: Amp <amp@ampcode.com>
|
||
|
KosinkadinkandAmp
|
1c8d341918 |
feat(capabilities): mirror missing audio/video/svg tokens, advertise round-trippable encodings
Pre-F1 immediate-follow-up #1 (client side): the client ``Capability`` mirror in ``protocol.py`` had drifted away from the server's ``comfy_rnp_protocol/constants.py`` — the audio sweep that landed ``AUDIO_MP3_BASE64`` / ``AUDIO_WAV_BASE64`` / ``AUDIO_WAV_INLINE``, the video sweep that landed ``VIDEO_MP4_BASE64``, and the SVG encoding ``SVG_XML_BASE64`` were declared on the server but never mirrored here. The mirror is supposed to be byte-equivalent so a client that imports ``Capability.AUDIO_MP3_BASE64`` doesn't crash when reading the same name a server descriptor emits. This PR adds the five missing tokens to the client mirror and expands ``CLIENT_CAPABILITIES`` to advertise the encodings the client actually round-trips today (audio MP3 / WAV decode, audio MP3 encode via ``encode_audio_input``, video MP4 base64 encode via ``encode_video_input``). ``SVG_XML_BASE64`` lands in the mirror but is **not** added to ``CLIENT_CAPABILITIES`` yet — there is no SVG envelope decoder in ``serialization.py`` today (Recraft SVG providers currently only round-trip via the ``IO.Custom`` opaque fallthrough Track §B added; a real ``decode_svg_envelope`` is a follow-up). Pairs with the matching ``feat/capability-mirror-sync`` PR on ``Comfy-Org/comfy-rnp-server`` (server-side capability gates: every descriptor that publishes ``"opaque"`` or ``"png_base64_batch"`` in its ``input_serialization`` now requires the matching token, so a client that doesn't advertise ``IO_OPAQUE`` / ``IMAGE_PNG_BASE64_BATCH`` is filtered out at descriptor-publish time); both PRs must merge for the gates to be useful end-to-end. ## Client - ``protocol.py``: add ``VIDEO_MP4_BASE64``, ``AUDIO_MP3_BASE64``, ``AUDIO_WAV_BASE64``, ``AUDIO_WAV_INLINE``, ``SVG_XML_BASE64`` to the ``Capability`` mirror class (byte-equivalent with server). - ``client.py``: add ``VIDEO_MP4_BASE64``, ``AUDIO_MP3_INLINE``, ``AUDIO_MP3_BASE64``, ``AUDIO_WAV_BASE64``, ``AUDIO_WAV_INLINE`` to ``CLIENT_CAPABILITIES`` so servers gating audio / video descriptors on these tokens hand them to this client. ``AUDIO_MP3_INLINE`` was implicitly supported by ``decode_audio_envelope`` but had never been advertised; now it is. ## Verification - ``serialization.encode_audio_input`` already emits ``mp3_base64`` envelopes (Stability audio sweep) — the client CAN produce this encoding, so advertising it is honest. - ``serialization.decode_audio_envelope`` already accepts ``("mp3_base64", "mp3_inline", "wav_base64", "wav_inline")`` — the client CAN consume all four, so advertising them is honest. - ``serialization.decode_video_envelope`` already accepts ``("mp4_inline", "mp4_base64")`` — both already advertised. - ``serialization.decode_envelope`` does not currently handle ``"svg"`` envelopes (no entry in ``_DECODERS``) so ``SVG_XML_BASE64`` is mirrored for byte-equivalence but intentionally NOT advertised. ## Lockstep - Client PR: this PR. - Server PR: ``feat/capability-mirror-sync`` on ``Comfy-Org/comfy-rnp-server`` (descriptor capability gates). - Both must merge; descriptor / advertisement-only change. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e29b4-d606-7278-8584-52910661c981 |
||
|
KosinkadinkandAmp
|
d67d42cd59 |
feat(custom-io): opaque pass-through custom IO bucket
Track B (opaque-custom-io) end-to-end on the proxy_node side. Adds a
generic ``IO.Custom(io_type)`` fallthrough in the parser so any
io_type string the client doesn't explicitly recognise becomes an
opaque socket with the original io_type preserved (so node-to-node
connections only chain when the strings match — an
``OPENAI_INPUT_FILES`` output cannot connect to a
``GEMINI_INPUT_FILES`` input). Pairs with the matching
``feat/opaque-custom-io`` PR on ``Comfy-Org/comfy-rnp-server`` (new
``Capability.IO_OPAQUE`` + the four deferred OpenAI / Gemini helper
node providers); both PRs must merge for the new path to light up
end-to-end.
## Client
- ``protocol.py``: mirror ``Capability.IO_OPAQUE = "io:opaque"``
from the server's ``comfy_rnp_protocol/constants.py``.
- ``client.py``: advertise ``Capability.IO_OPAQUE`` in
``CLIENT_CAPABILITIES`` so servers gating descriptors on the
token (every helper-config descriptor declares
``required_capabilities: [Capability.IO_OPAQUE]``) hand them to
this client.
- ``proxy_node._parse_input_spec``: after the explicit STRING / INT
/ FLOAT / BOOLEAN / COMBO / IMAGE / VIDEO / AUDIO / MASK branches,
any non-empty string io_type falls through to
``IO.Custom(io_type).Input(name, optional=…, tooltip=…)``. The
V3 ``@comfytype`` decorator binds the original io_type string to
the synthesised ComfyType, so the frontend's connection-validity
check enforces strict string matching between sockets.
- ``proxy_node._parse_outputs``: mirror branch — when
``_OUTPUT_CLASSES.get(io_type)`` is None, build
``IO.Custom(io_type).Output(...)`` instead of failing the whole
descriptor. Only a non-string / empty io_type still warns and
skips.
- Encoder dispatch: opaque values flow through ``_encode_one``
unchanged (already JSON-serialisable dicts/lists/strings — no
envelope wrapping). Verified against the existing fallthrough at
the end of ``_encode_one`` (returns ``value`` for non-tensor /
non-audio inputs). Output deserialisation already passes
non-envelope values through unchanged in
``_deserialize_output``.
## Docs
The proxy_node module-level docstring is updated to reflect the
new fallthrough (previously documented only the strict-skip
behaviour for unknown io_types).
## Verification
- Static AST parse of all four touched modules
(``proxy_node.py``, ``client.py``, ``serialization.py``,
``protocol.py``).
- ``Capability.IO_OPAQUE`` resolves to ``"io:opaque"`` (matches the
server's wire shape).
- ``IO.Custom("OPENAI_INPUT_FILES").Input("files", optional=True)``
and ``.Output()`` both expose ``io_type ==
"OPENAI_INPUT_FILES"``; ``IO.Custom("GEMINI_INPUT_FILES")``
similarly exposes its own io_type, so the two sockets cannot
cross-chain.
## Lockstep
Pairs with ``feat/opaque-custom-io`` on
``Comfy-Org/comfy-rnp-server`` (the new ``Capability.IO_OPAQUE``
constant lives there as the source of truth; this repo's
``protocol.py`` is the byte-compatible mirror). The server PR also
ports the four deferred helper nodes that exercise the new path
end-to-end (``OpenAIInputFiles_RNP``, ``OpenAIChatConfig_RNP``,
``GeminiInputFiles_RNP``, plus the chat node's ``files=`` /
``advanced_options=`` inputs). Until that PR merges, this client
parser fallthrough lights up only against the existing Recraft
custom IO descriptors (``RECRAFT_COLOR``, ``RECRAFT_CONTROLS``,
``RECRAFT_V3_STYLE``) — which previously caused the whole
descriptor to be skipped because the parser had no fallthrough.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e29b4-d606-7278-8584-52910661c981
|
||
|
KosinkadinkandAmp
|
94f995bb05 |
feat(async-polling): advertise ASYNC_EXECUTE capability and derive poll cap from duration
Final pieces of Track A (async-polling mode) on the client side. The proxy_node already routes between ``/execute`` and ``/execute_async`` on the descriptor's ``remote.execution.mode``; this commit makes the contract honest in two ways: - ``client.py``: advertise the new ``Capability.ASYNC_EXECUTE`` token so the server can gate ``async_polling`` descriptors on its presence. An older client that only knows ``/execute`` is now visible to the server and won't be silently routed to a long-poll provider whose lifecycle it can't drive. - ``protocol.py`` + ``proxy_node.py``: drop the ``DEFAULT_MAX_POLL_ATTEMPTS`` wire-default in favour of duration-based ``DEFAULT_SOFT_TIMEOUT_S`` / ``DEFAULT_HARD_TIMEOUT_S`` / ``DEFAULT_POLL_GRACE_S``. The poll-loop cap is now a *derived* value (``ceil((hard_timeout_s + grace) / poll_interval_s)``) so changing the poll cadence can never silently shrink or stretch the contract. Industry precedent: OAuth2 device flow §3.5, Kubernetes ``--timeout``, gRPC deadlines all express bounds as durations. Lockstep with the server side of Track A (comfy-rnp-server feat/async-polling-mode), which now enforces ``mode`` dispatch on ``/execute`` vs ``/execute_async`` and migrates existing providers (BFL/Bria/LTXV/Grok video/OpenAIChatNode/Stability UpscaleCreative → ``async_polling``; DALL-E/Ideogram/Recraft/sync Stability → ``request_response``). Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e28eb-31ae-76a1-bb45-c0922f43bcc3 |
||
|
Jedrzej KosinskiandAmp
|
ed63c5ae49 |
feat(serialization): decode image:png_base64_batch envelope
Mirrors the server-side change in comfy-rnp-server PR #7. Adds the client-side half of the multi-image envelope contract: - Capability.IMAGE_PNG_BASE64_BATCH (\image:png_base64_batch\) added to protocol.Capability, mirroring comfy_rnp_protocol/constants.py. - decode_image_envelope() now dispatches png_base64_batch to a new _decode_image_batch_envelope() helper that stacks each PNG frame into a [B, H, W, C] tensor — the native ComfyUI image-batch shape — so providers whose upstream returns n>1 surface as a real batched tensor in the workflow runtime instead of being silently truncated to the first frame. - Frame-count integrity check: if the envelope declares shape[0]=B but \ rames\ has a different length, fail hard rather than drop frames silently. Matches the SVG decoder's \count\ check. - Per-frame H/W mismatch is also a hard error — the server-side encoder enforces uniform sizing already, so a mismatch on the wire indicates a wire bug worth surfacing. - IMAGE_PNG_BASE64_BATCH added to client.CLIENT_CAPABILITIES so the server's per-request negotiator (\client_advertises_capability\ in comfy_rnp_protocol.capabilities) lights up the batch path. Single-frame envelopes (\png_base64\ / \jpeg_base64\ / \webp_base64\) remain unchanged — back-compat with every already-merged provider. Verified end-to-end with a synthetic 2-frame envelope (white + black 32x64 PNGs) plus the mismatch / empty-frames error paths. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e24d1-250c-750a-ab00-2eb5e9cb2500 |
||
|
Kosinkadink
|
cf55bd58ae |
Initial RNP/1 client implementation with Tier 3 features
ComfyUI custom-node package that fetches node descriptors from an
RNP/1 (Remote Node Protocol v1) server and exposes them as native V3
nodes inside ComfyUI.
Includes:
- Manifest + capability handshake, ETag-conditional object_info fetch
- Sync execute_remote and async _run_async_polling flows via
comfy_api_nodes.util.client.poll_op_raw
- Image / mask / audio / video envelope encode + decode
- Presigned upload helper for >8 MiB inputs
- Tier 3:
* TASK_LOST detection + idempotent resume — survives an RNP server
restart mid-task
* SERVER_BUSY / MAINTENANCE 503 envelopes treated as rate limits
with Retry-After honoring and friendly per-second progress text
* Out-of-band url_fetch for URI-only output envelopes per the
descriptor's remote.url_fetch policy
* Per-poll auth refresh so long-running tasks pick up rotated tokens
Pairs with Comfy-Org/comfy-rnp-server (private) as the reference
server implementation.
|