37a4961d6d61e8a1d08fcb2356fc858086d878c3
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 KosinkadinkandAmp
|
37a4961d6d |
feat(local-validate): fail-fast on over-cap image batches
Wire ``proxy_node._encode_inputs`` to consult each input's
``descriptor.input.<section>.<name>[1].local_validate.image_max_batch``
rule. When set, the encoder asserts ``tensor.shape[0] <= cap`` for
IMAGE inputs (rank-4 ``[B,H,W,C]`` tensors) and raises
``RnpProtocolError(code=INPUT_INVALID, user_facing=True)`` *before*
any HTTP round trip — so a workflow with N>cap reference frames
surfaces the per-provider error in the standard friendly-message
dialog instead of paying for a doomed upstream call.
## Server
(See companion PR ``Comfy-Org/comfy-rnp-server#15`` —
``feat(descriptor): declare image_max_batch local_validate caps``.)
The server-side per-call caps inside each provider's ``execute()``
stay in place as defense-in-depth; non-compliant clients still fail.
## Client
- ``proxy_node._collect_local_validate(descriptor)`` walks
``descriptor.input.{required,optional}`` and returns
``{name: rules}`` for every input that carries a non-empty
``local_validate`` block. Inputs nested inside ``AUTOGROW`` /
``DYNAMIC_COMBO`` aren't surfaced — the parser doesn't yet flatten
them into top-level inputs, so the cap declared on those templates
documents the future contract rather than firing today.
- ``proxy_node._enforce_local_validate(name, value, rules,
*, node_id)`` applies each supported rule to ``value``. Today only
``image_max_batch`` is implemented: rank-4 / rank-3-channels-last
tensors with ``shape[0] > cap`` raise ``RnpProtocolError`` with
code ``INPUT_INVALID`` and ``user_facing=True``. Friendly per-
provider message echoes the upstream wording (Flux2's "The
current maximum number of supported images is N" for cap≥2;
a "expects a single image frame" copy for cap=1 like LTXV i2v).
- ``_encode_inputs`` runs the validator before the encoder so the
raise happens before any tensor → PNG conversion or
``maybe_externalize`` upload — the cheapest possible fail point.
- Lookup is defensive at every layer: missing ``local_validate`` →
no-op, missing ``image_max_batch`` sub-key → no-op, non-tensor
input → no-op, non-image rank → no-op. Older descriptors emit
no rule and the encoder behaves identically to before.
## Docs
(Matrix prose updated on the server side — the canonical
``notes/remote-nodes-port-matrix.md`` lives in the rnp-server repo.)
## Verification
- ``notes/run_image_max_batch.py`` — new standalone smoke test
(loads the helper functions via AST extraction so the test runs
in a bare venv with just ``torch`` installed; doesn't drag in
the ComfyUI package). Asserts:
1. ``_collect_local_validate`` extracts the rule from a Flux2-
shaped descriptor.
2. A 3-frame tensor against ``cap=2`` raises ``RnpProtocolError``
with ``code=INPUT_INVALID``, ``user_facing=True``, and the
``node_id`` + counts mentioned in the message.
3. A 2-frame tensor against ``cap=2`` encodes cleanly.
4. ``cap=1`` surfaces the single-frame copy.
5. Descriptor without ``local_validate`` is a no-op.
6. ``local_validate`` block missing the ``image_max_batch`` key
is a no-op.
Run with: ``python notes/run_image_max_batch.py`` — all
assertions pass on PASS.
## Lockstep
Paired with server PR ``Comfy-Org/comfy-rnp-server#15``
(``feat/image-max-batch-rule``). Both branches were cut off
``origin/main`` before PR-2 lands; the orchestrator will rebase PR-3
onto PR-2 before merging.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e2adb-6ddf-72ad-88e1-8b30c57e873f
|
||
|
Jedrzej Kosinski
|
6a82114ded |
feat(serialization): input-side image:png_base64_batch envelope (#7)
feat: input-side image:png_base64_batch envelope |
||
|
KosinkadinkandAmp
|
aad9f0ba43 |
feat(serialization): input-side image:png_base64_batch envelope
Closes the silent `tensor[0]` truncation bug for multi-image inputs:
when a descriptor advertises `"png_base64_batch"` for an IMAGE input
and the inbound tensor has `shape[0] > 1`, the encoder now emits a
multi-frame envelope (`frames=[…]`) instead of dropping `tensor[1:]`.
Lockstep server change ships in `comfy-rnp-server` PR (paired branch
`feat/input-batch-envelope`).
## Client
- `serialization.py`: extend `encode_image_tensor` with an
`accepts_batch=False` keyword. When True AND `tensor.shape[0] > 1`,
emit `{"type":"image","encoding":"png_base64_batch","frames":[…],
"shape":[B,H,W,C],…}` — the same wire shape the existing
`_decode_image_batch_envelope` consumes on the response path, so
both directions round-trip symmetrically. Default-False keeps the
legacy single-frame contract for descriptors that haven't migrated.
- `proxy_node.py`: plumb the per-input `input_serialization`
allow-list from `descriptor.remote.input_serialization` through
`build_node_class` → `_execute_remote` → `_encode_inputs` →
`_encode_one`. The wire shape `{input_name: <str | list[str]>}`
normalizes to `dict[str, list[str]]` once at descriptor parse time
so the encoder can do a single `in` check per input. `_encode_one`
passes `accepts_batch=("png_base64_batch" in encodings)` to the
image encoder.
- `IMAGE_PNG_BASE64_BATCH` is already in `client.CLIENT_CAPABILITIES`
(was added when the output-side decoder shipped) so no wire-header
change.
## Verification
- Both modified files parse cleanly via `ast.parse`.
- `serialization._decode_image_batch_envelope` (output-side decoder
already in this file) consumes the same `frames=[…]` shape the
new encoder emits — round-trips symmetrically.
## Lockstep
- Paired server PR: `comfy-rnp-server` branch
`feat/input-batch-envelope` — adds `"png_base64_batch"` to
`DEFAULT_INPUT_SERIALIZATIONS["image"]`, the shared
`comfy_rnp_protocol.envelopes.resolve_image_envelope_pngs(envelope)`
helper, and migrates Flux2Pro/Max, OpenAI ChatNode + GPTImageV2,
Grok ImageEditV2 + VideoReference, and LTXV ImageToVideo
providers to consume the multi-frame envelope. Both PRs must
merge together; no back-compat between unmatched versions.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e29b4-d606-7278-8584-52910661c981
|
||
|
KosinkadinkandAmp
|
d67d42cd59 |
feat(custom-io): opaque pass-through custom IO bucket
Track B (opaque-custom-io) end-to-end on the proxy_node side. Adds a
generic ``IO.Custom(io_type)`` fallthrough in the parser so any
io_type string the client doesn't explicitly recognise becomes an
opaque socket with the original io_type preserved (so node-to-node
connections only chain when the strings match — an
``OPENAI_INPUT_FILES`` output cannot connect to a
``GEMINI_INPUT_FILES`` input). Pairs with the matching
``feat/opaque-custom-io`` PR on ``Comfy-Org/comfy-rnp-server`` (new
``Capability.IO_OPAQUE`` + the four deferred OpenAI / Gemini helper
node providers); both PRs must merge for the new path to light up
end-to-end.
## Client
- ``protocol.py``: mirror ``Capability.IO_OPAQUE = "io:opaque"``
from the server's ``comfy_rnp_protocol/constants.py``.
- ``client.py``: advertise ``Capability.IO_OPAQUE`` in
``CLIENT_CAPABILITIES`` so servers gating descriptors on the
token (every helper-config descriptor declares
``required_capabilities: [Capability.IO_OPAQUE]``) hand them to
this client.
- ``proxy_node._parse_input_spec``: after the explicit STRING / INT
/ FLOAT / BOOLEAN / COMBO / IMAGE / VIDEO / AUDIO / MASK branches,
any non-empty string io_type falls through to
``IO.Custom(io_type).Input(name, optional=…, tooltip=…)``. The
V3 ``@comfytype`` decorator binds the original io_type string to
the synthesised ComfyType, so the frontend's connection-validity
check enforces strict string matching between sockets.
- ``proxy_node._parse_outputs``: mirror branch — when
``_OUTPUT_CLASSES.get(io_type)`` is None, build
``IO.Custom(io_type).Output(...)`` instead of failing the whole
descriptor. Only a non-string / empty io_type still warns and
skips.
- Encoder dispatch: opaque values flow through ``_encode_one``
unchanged (already JSON-serialisable dicts/lists/strings — no
envelope wrapping). Verified against the existing fallthrough at
the end of ``_encode_one`` (returns ``value`` for non-tensor /
non-audio inputs). Output deserialisation already passes
non-envelope values through unchanged in
``_deserialize_output``.
## Docs
The proxy_node module-level docstring is updated to reflect the
new fallthrough (previously documented only the strict-skip
behaviour for unknown io_types).
## Verification
- Static AST parse of all four touched modules
(``proxy_node.py``, ``client.py``, ``serialization.py``,
``protocol.py``).
- ``Capability.IO_OPAQUE`` resolves to ``"io:opaque"`` (matches the
server's wire shape).
- ``IO.Custom("OPENAI_INPUT_FILES").Input("files", optional=True)``
and ``.Output()`` both expose ``io_type ==
"OPENAI_INPUT_FILES"``; ``IO.Custom("GEMINI_INPUT_FILES")``
similarly exposes its own io_type, so the two sockets cannot
cross-chain.
## Lockstep
Pairs with ``feat/opaque-custom-io`` on
``Comfy-Org/comfy-rnp-server`` (the new ``Capability.IO_OPAQUE``
constant lives there as the source of truth; this repo's
``protocol.py`` is the byte-compatible mirror). The server PR also
ports the four deferred helper nodes that exercise the new path
end-to-end (``OpenAIInputFiles_RNP``, ``OpenAIChatConfig_RNP``,
``GeminiInputFiles_RNP``, plus the chat node's ``files=`` /
``advanced_options=`` inputs). Until that PR merges, this client
parser fallthrough lights up only against the existing Recraft
custom IO descriptors (``RECRAFT_COLOR``, ``RECRAFT_CONTROLS``,
``RECRAFT_V3_STYLE``) — which previously caused the whole
descriptor to be skipped because the parser had no fallthrough.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019e29b4-d606-7278-8584-52910661c981
|
||
|
KosinkadinkandAmp
|
94f995bb05 |
feat(async-polling): advertise ASYNC_EXECUTE capability and derive poll cap from duration
Final pieces of Track A (async-polling mode) on the client side. The proxy_node already routes between ``/execute`` and ``/execute_async`` on the descriptor's ``remote.execution.mode``; this commit makes the contract honest in two ways: - ``client.py``: advertise the new ``Capability.ASYNC_EXECUTE`` token so the server can gate ``async_polling`` descriptors on its presence. An older client that only knows ``/execute`` is now visible to the server and won't be silently routed to a long-poll provider whose lifecycle it can't drive. - ``protocol.py`` + ``proxy_node.py``: drop the ``DEFAULT_MAX_POLL_ATTEMPTS`` wire-default in favour of duration-based ``DEFAULT_SOFT_TIMEOUT_S`` / ``DEFAULT_HARD_TIMEOUT_S`` / ``DEFAULT_POLL_GRACE_S``. The poll-loop cap is now a *derived* value (``ceil((hard_timeout_s + grace) / poll_interval_s)``) so changing the poll cadence can never silently shrink or stretch the contract. Industry precedent: OAuth2 device flow §3.5, Kubernetes ``--timeout``, gRPC deadlines all express bounds as durations. Lockstep with the server side of Track A (comfy-rnp-server feat/async-polling-mode), which now enforces ``mode`` dispatch on ``/execute`` vs ``/execute_async`` and migrates existing providers (BFL/Bria/LTXV/Grok video/OpenAIChatNode/Stability UpscaleCreative → ``async_polling``; DALL-E/Ideogram/Recraft/sync Stability → ``request_response``). Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019e28eb-31ae-76a1-bb45-c0922f43bcc3 |
||
|
Kosinkadink
|
cf55bd58ae |
Initial RNP/1 client implementation with Tier 3 features
ComfyUI custom-node package that fetches node descriptors from an
RNP/1 (Remote Node Protocol v1) server and exposes them as native V3
nodes inside ComfyUI.
Includes:
- Manifest + capability handshake, ETag-conditional object_info fetch
- Sync execute_remote and async _run_async_polling flows via
comfy_api_nodes.util.client.poll_op_raw
- Image / mask / audio / video envelope encode + decode
- Presigned upload helper for >8 MiB inputs
- Tier 3:
* TASK_LOST detection + idempotent resume — survives an RNP server
restart mid-task
* SERVER_BUSY / MAINTENANCE 503 envelopes treated as rate limits
with Retry-After honoring and friendly per-second progress text
* Out-of-band url_fetch for URI-only output envelopes per the
descriptor's remote.url_fetch policy
* Per-poll auth refresh so long-running tasks pick up rotated tokens
Pairs with Comfy-Org/comfy-rnp-server (private) as the reference
server implementation.
|