feat: integrate native image-list caching and exact serverless value overrides
This commit is contained in:
Robert Wojciechowski
@@ -141,13 +141,16 @@ Also, for collector-based flows:
|
||||
|
||||
## Endpoint: `POST /distributed/job_complete`
|
||||
|
||||
Submit one completed worker image back to the master collector queue.
|
||||
Submit direct-worker image/audio results to the master Collector queue. The route
|
||||
accepts canonical JSON completions or multipart image chunks, including opt-in
|
||||
exact floating-point IMAGE transport. Serverless typed results use their separate
|
||||
bridge/provider contract.
|
||||
|
||||
### URL
|
||||
|
||||
- `http://<master-host>:<master-port>/distributed/job_complete`
|
||||
|
||||
### Request Body
|
||||
### JSON request body
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -164,13 +167,35 @@ Submit one completed worker image back to the master collector queue.
|
||||
- `job_id` (string, required)
|
||||
- `worker_id` (string, required)
|
||||
- `batch_idx` (integer >= 0, required)
|
||||
- `image` (string, required)
|
||||
- `image` (string, optional if `audio` is present)
|
||||
- PNG payload as either:
|
||||
- data URL: `data:image/png;base64,...`
|
||||
- raw base64 PNG bytes
|
||||
- `is_last` (boolean, required)
|
||||
|
||||
Legacy multipart/tensor payload formats are no longer accepted on this endpoint.
|
||||
At least one of `image` or `audio` must be present. Optional `audio` is an object
|
||||
with `sample_rate` (positive integer), `shape` (`[batch, channels, samples]`),
|
||||
`dtype: "float32"`, and `data` (base64 waveform bytes). Audio-only completions
|
||||
still include `batch_idx`; the Collector uses `0` and `is_last: true`.
|
||||
|
||||
### Multipart image chunks
|
||||
|
||||
Send `multipart/form-data` with a JSON `envelope` field and repeated binary
|
||||
`media` file fields. The envelope contains `job_id`, `worker_id`, `format`,
|
||||
`start_index` (nonnegative integer), `count`, `width`, `height`, `is_last`, and
|
||||
optional canonical `audio`.
|
||||
|
||||
- `png`, `jpeg`, `webp`: one `media` part per frame.
|
||||
- `ffv1`, `h264`, `h265` (experimental), `av1`: one video part per chunk.
|
||||
- `exact`: one raw tensor part per frame and an envelope `descriptors` list with
|
||||
one `tensor-le-v1` IMAGE descriptor per part. Each descriptor shape must be
|
||||
`[1, height, width, channels]`; decoding preserves supported float dtype and
|
||||
values instead of quantizing to 8-bit.
|
||||
|
||||
`legacy_png` uses JSON, not multipart. Exact transport requires support on both
|
||||
sides and never retries as an 8-bit format when an older master rejects it.
|
||||
See [result transfer](compressed-result-transfer.md) for format controls,
|
||||
non-exact compatibility fallbacks and request limits.
|
||||
|
||||
### CORS note
|
||||
|
||||
|
||||
@@ -1,38 +1,49 @@
|
||||
# Compressed Result Transfer
|
||||
# Compressed and Exact Result Transfer
|
||||
|
||||
**Distributed Collector** has two optional widgets that control how workers send their images back to the master. This matters most when workers run in the cloud and you pay for data leaving it (egress).
|
||||
**Distributed Collector** (`DistributedCollector`) collects images and/or audio. Its `transfer_format` and `transfer_quality` widgets control **remote image returns only**; they do not re-encode master-local images or change audio transport. The saved defaults remain `legacy_png` and quality `90`.
|
||||
|
||||
| `transfer_format` | Type | Notes |
|
||||
| `transfer_format` | Fidelity | Direct-worker transport |
|
||||
|---|---|---|
|
||||
| `legacy_png` (default) | lossless | Original behaviour: one JSON request per frame, uncompressed PNG in base64. Works with any master version. |
|
||||
| `png` | lossless | Compressed PNG, binary, several frames per request. Same tensors as `legacy_png`. |
|
||||
| `ffv1` | lossless | Whole batch as one FFV1 video. Same tensors as `legacy_png`; best lossless choice for frame sequences. |
|
||||
| `jpeg`, `webp` | lossy | Per-frame stills. Good for image batches. |
|
||||
| `h264`, `h265`, `av1` | lossy | Whole batch as one video. By far the smallest for video frame batches. |
|
||||
| `legacy_png` (default) | 8-bit pixels | Original behaviour: one JSON request per frame, uncompressed PNG in base64. Compatible with legacy masters. |
|
||||
| `png` | Lossless 8-bit pixels | Compressed PNG, binary, several frames per request. |
|
||||
| `ffv1` | Lossless 8-bit pixels | FFV1 video chunks; useful for frame sequences. |
|
||||
| `jpeg`, `webp` | Lossy 8-bit images | Per-frame stills. |
|
||||
| `h264`, `h265` (**experimental**), `av1` | Lossy 8-bit images | Video chunks; useful for similar consecutive frames. |
|
||||
| `exact` (opt-in) | Original floating-point tensors | Binary multipart chunks with one raw tensor payload and `tensor-le-v1` descriptor per frame. No 8-bit quantization. |
|
||||
|
||||
`transfer_quality` (1–100, default 90) applies to the lossy formats. JPEG/WebP use it directly; video codecs map it to CRF (90 → CRF 13 for h264/h265, CRF 16 for av1; 100 → CRF 0).
|
||||
“Lossless” PNG/FFV1 means lossless **after 8-bit quantization**, not preservation of the source float tensor. Direct non-exact returns decode to RGB float32. `exact` preserves supported floating-point dtype, shape, channels and values: float16, bfloat16, float32 or float64, finite dense BHWC tensors with 1, 3 or 4 channels. It generally costs substantially more bandwidth.
|
||||
|
||||
Measured on a 121-frame 1280×720 batch (one worker's result):
|
||||
`transfer_quality` (1–100, default 90) applies to lossy formats, not `exact`, PNG or FFV1. JPEG/WebP use it directly; video codecs map it to CRF (90 → CRF 13 for h264/h265, CRF 16 for av1; 100 → CRF 0). A quality of 100 is not a substitute for exact float transport.
|
||||
|
||||
## Serverless profiles
|
||||
|
||||
- **`comfy-media-v1`:** the same Collector supplies the endpoint's image-return policy. The saved `legacy_png` default is mapped to compressed **`png`** with the same 8-bit pixel precision; normal serverless returns are not exact by default. All other formats retain their selection, including opt-in `exact` and experimental `h265`. Missing codecs or rejected formats fail instead of selecting another codec. Typed image returns preserve channel count and carry alpha separately where needed. See [serverless images and audio](serverless-media.md).
|
||||
- **`legacy-image-v1`:** retains its image-file/`SaveImage` contract, not the typed return-format controls. Selecting a legacy pool when any Collector requests `exact` is rejected before dispatch; there is no 8-bit fallback. Audio requires direct workers or `comfy-media-v1`.
|
||||
|
||||
## Direct-worker compatibility and limits
|
||||
|
||||
- Choose the format on the master; the value is sent to workers with the workflow.
|
||||
- **`exact` requires support on both master and worker.** If an older master rejects an exact chunk, the worker raises an error and never resends it as `legacy_png` or another 8-bit format. An exact Collector also rejects direct-worker image returns labelled with a non-exact format. Update both sides or explicitly select a non-exact format.
|
||||
- For **non-exact compressed** formats only, the specific first-request older-master rejection (`400` with `Invalid JSON payload`) causes a retry using `legacy_png`. Other rejections, including HTTP 413, remain errors.
|
||||
- For direct video returns only, an unavailable encoder falls back to `h264` for lossy video if available, otherwise `png`; unavailable `ffv1` falls back to `png`. The worker logs this. This compatibility fallback does **not** apply to `exact` or typed serverless returns.
|
||||
- Multipart requests use a 32 MiB chunk target (`COMFYUI_DISTRIBUTED_TRANSFER_CHUNK_BYTES`) and a 96 MiB request budget (`COMFYUI_DISTRIBUTED_TRANSFER_MAX_REQUEST_BYTES`), reserving space for the envelope and accounting separately for audio. A single large frame may exceed the chunk target but must fit the request budget. Keep the request budget below the master's `--max-upload-size` (default 100 MB).
|
||||
- Non-exact compressed chunks are all encoded before sending. Exact transfer instead plans frame sizes from shape/dtype before sending and encodes one chunk at a time. An oversized individual frame is rejected before any chunks are sent; choose another format or raise the worker and master limits together.
|
||||
- Audio is attached to the final image chunk when it fits, otherwise sent separately. Audio-only completions use the canonical JSON envelope. Direct audio uses float32 waveform bytes regardless of the image format; `exact` is an IMAGE option, not a new audio codec.
|
||||
- Each worker logs the format and bytes sent. These request limits are for direct multipart transfers; serverless request/artifact limits are documented separately.
|
||||
|
||||
## Previously documented size example
|
||||
|
||||
The following measurements are retained from a 121-frame 1280×720 slow-pan batch (one worker's result); they are not new validation of the current implementation or exact transport.
|
||||
|
||||
| Format | Sent | vs legacy | PSNR |
|
||||
|---|---|---|---|
|
||||
| `legacy_png` | 446 MB | 1× | lossless |
|
||||
| `png` | 163 MB | 2.7× | lossless |
|
||||
| `ffv1` | 128 MB | 3.5× | lossless |
|
||||
| `legacy_png` | 446 MB | 1× | lossless at 8-bit precision |
|
||||
| `png` | 163 MB | 2.7× | lossless at 8-bit precision |
|
||||
| `ffv1` | 128 MB | 3.5× | lossless at 8-bit precision |
|
||||
| `jpeg` q90 | 27 MB | 17× | 42.7 dB |
|
||||
| `webp` q90 | 13 MB | 33× | 40.6 dB |
|
||||
| `h264` q90 | 2.4 MB | 186× | 40.2 dB |
|
||||
| `h265` q90 | 1.0 MB | 434× | 40.1 dB |
|
||||
| `h265` q90 (experimental) | 1.0 MB | 434× | 40.1 dB |
|
||||
| `av1` q90 | 0.4 MB | 1031× | 40.5 dB |
|
||||
|
||||
The test clip is a slow pan, which video codecs compress especially well; expect smaller (still large) gains on content with more motion.
|
||||
|
||||
Notes:
|
||||
- Choose the format on the master; the value is sent to workers with the workflow.
|
||||
- The master must run this version to accept compressed results. A worker sending to an older master gets a 400 on its first request and automatically resends with `legacy_png`.
|
||||
- If a worker's PyAV build lacks the requested video encoder, it falls back to `h264` (lossy formats) or `png` (`ffv1`), and logs it.
|
||||
- Lossy formats only affect frames produced by workers; the master's own frames are never re-encoded.
|
||||
- Request size is bounded on the final encoded bytes. Frames are batched into requests of up to 32 MiB (`COMFYUI_DISTRIBUTED_TRANSFER_CHUNK_BYTES`); a single large frame may go alone in a request of up to 96 MiB (`COMFYUI_DISTRIBUTED_TRANSFER_MAX_REQUEST_BYTES`). Keep the latter below the master's `--max-upload-size` (default 100 MB) if you change either.
|
||||
- All chunks are encoded before anything is sent. If a single frame cannot fit within the request limit, the worker fails with an error and sends nothing; use a lossy format or raise both limits.
|
||||
- A rejection other than the older-master case above (for example the master's size limit, HTTP 413) is reported as an error rather than retried as `legacy_png`.
|
||||
- Each worker logs what it sent, e.g. `Sent 121 image(s) as h264 in 1 request(s): 2.40 MB (raw 334.49 MB, 139.4x smaller)`.
|
||||
Video codecs compress this slow pan especially well; gains depend on content and motion.
|
||||
|
||||
@@ -52,7 +52,31 @@ Start outputs, in order: `image`, `mask`, `item_index` (zero-based), `context`,
|
||||
|
||||
Values that cannot be transported exactly (models, latents, conditioning and other objects) are rejected. Load models inside the region from a loader connected directly to the processing nodes, as described below.
|
||||
|
||||
End accepts `images`, optional `masks` / `texts`, and `context`. It returns `image_list`, `masks`, and `texts` in original item order. Each job must produce exactly one image, at most one mask, and at most one text value. Unconnected optional outputs contain `None` entries.
|
||||
End accepts `images`, optional `masks` / `texts`, and `context`. Its first three outputs remain `image_list`, `masks`, and `texts` in original item order. Each job must produce exactly one image, at most one mask, and at most one text value. Unconnected optional mask/text outputs contain `None` entries.
|
||||
|
||||
### Keep values through End without extra wires
|
||||
|
||||
Start's existing **context** connection now also lets End recover each item's original `value_1` … `value_10`. You do not need to wire those values individually into End. Context remains a lightweight execution reference, not a general-purpose data pipe.
|
||||
|
||||
End appends a **values** output (`DISTRIBUTED_IMAGE_LIST_VALUES`): one value bundle per image, in the same source order. Connect it to **Distributed Image List Get Value**, then choose `value_name` to extract the field you need. The getter preserves its type and ComfyUI maps it across the bundles, producing a list aligned with End's images. Missing fields fail clearly instead of producing empty placeholders.
|
||||
|
||||
For example, to preserve filenames:
|
||||
|
||||
```text
|
||||
filenames → Start.value_1
|
||||
Start.context ─────────────→ End.context
|
||||
Start.image → processing → End.images
|
||||
├─ image_list → Save.images
|
||||
└─ values → Get Value [value_1] → Save.filename_prefix
|
||||
```
|
||||
|
||||
**New or changed per-item values:** connect the calculated result to End's optional `value_N` input. It replaces the original field with that number, or adds it if Start did not supply it. Every connected override must produce exactly one supported value for each item. A tensor batch remains one value, just as it does at Start. Image, mask, string, integer, float and boolean values are supported; arbitrary objects/models/latents and non-finite numbers are not.
|
||||
|
||||
End shows override inputs progressively. Use **Add value override** to expose a later slot without connecting the earlier ones. Disconnecting an override never renumbers the other End fields: `value_3` always means `value_3`. Unlike Start's native Autogrow inputs, End overrides must keep these identities stable. Existing image/mask/text/context sockets and output indices are unchanged. If the frontend extension is unavailable, Python still declares all optional override inputs and the values output.
|
||||
|
||||
Original values are retained on the master; workers only return explicitly connected overrides, avoiding a redundant return trip for unchanged reference images or other values. Overrides use the existing exact value transport and size limits. Selected workers must advertise `value_results: true` when overrides are requested, checked before execution and again at dispatch. Original-value passthrough alone does not require that additional capability.
|
||||
|
||||
Values leaving the region must come from **End**, not directly from Start. To chain regions, extract an End value and connect it to the next Start's corresponding input. Values remain available downstream after End cleans up the execution context; they do not depend on a later lookup of expired job state.
|
||||
|
||||
Models are loaded and cached on each processing participant, not serialized between machines. Connect loader outputs directly to the processing nodes. The master needs the region's model files when participating, as does each selected worker. The master still validates the prompt: it needs the relevant node definitions and any model-name/file availability required by their validation (including combo lists). This is not a model-free-master guarantee.
|
||||
|
||||
@@ -116,6 +140,7 @@ Provider health/authentication probing does not verify bridge/model installation
|
||||
IMAGE returns are exact by default, or explicitly encoded as selected at End.
|
||||
MASK tensor data, input images and text retain exact typed codecs. There is no
|
||||
image-list-specific endpoint protocol or new visible node socket.
|
||||
- Connected End `value_1` … `value_10` overrides are computed on the endpoint and returned using exact typed `VALUE` exports. Sparse slot identities are preserved; only explicitly connected overrides are transported. The master overlays them on each source item’s original values. BOOLEAN, INT, FLOAT, STRING, IMAGE and MASK remain distinct; missing/extra slots, multiple values and nonfinite values fail. Override tensors remain exact even when the main image uses a compressed return format. End overrides require the updated complete bridge (including `value_codec.py`), not merely an older healthy media endpoint.
|
||||
- Cloud `value_N` tensors must have native IMAGE `[B,H,W,C]` or MASK `[B,H,W]`
|
||||
shapes, not arbitrary rank-one/rank-two tensor payloads. Typed tensors must be
|
||||
finite; connected masks must match their image grid. Unsupported values fail
|
||||
@@ -134,10 +159,22 @@ Provider health/authentication probing does not verify bridge/model installation
|
||||
before reuse. Preserve its retained files until execution and readers are known
|
||||
to have stopped; see the [bridge uncertainty policy](../serverless_bridge/README.md).
|
||||
|
||||
## Native cache reuse
|
||||
|
||||
Repeated items can reuse **ComfyUI's native processing cache** on the same participant. Start uses a deterministic identity of the exact image, optional mask, all supplied value slots and source index; tensor dtype, shape and bytes are preserved. Fresh job IDs, assignment tokens and callback addresses live at End, so they no longer invalidate otherwise unchanged upstream processing. End still validates the current assignment and delivers results in source order on every run.
|
||||
|
||||
- Processing settings, seeds, links and node fingerprints still participate in ComfyUI's normal signatures. Changed content invalidates reuse; nodes that declare themselves noncacheable still run.
|
||||
- The master's private executor retains one native LRU output cache between successful slots, capped at **128 entries when retained**. An active graph may temporarily exceed that target; oversized generations are not retained. Failed or cancelled local slots do not seed reuse, and `--cache-none` disables the local output cache. Direct workers use their own configured native caches.
|
||||
- Cache hits are best-effort, not guaranteed: eviction, restarts, changed ordering/index, changed settings and assignment to a different worker can cause misses. There is no worker affinity, disk cache or separate completed-result store.
|
||||
- Reuse retains tensor/model references and can increase RAM/VRAM use; the entry cap is **not a byte limit**. Exact identity calculation also adds CPU hashing and, for GPU-resident inputs, a CPU copy.
|
||||
- Remote items are still uploaded. Each upload returns a worker-computed identity; if a compatible older worker does not return a valid identity, that assignment uses the legacy graph without stable-input reuse. A worker downgrade between items also falls back safely. Assignment expiry and current-item validation apply even when processing hits the cache.
|
||||
- Keep side-effect nodes outside the region when they must run on every submission. A cached processing node follows ordinary ComfyUI semantics and does not repeat its save/preview side effects; End's delivery and downstream saving still run.
|
||||
|
||||
## Scope and failure behavior
|
||||
|
||||
- This version supports master, direct workers and `comfy-media-v1` serverless pools in one image-list region. Direct capability/idle HTTP probes are never sent to serverless pools; pool probing uses the existing serverless health/authentication path.
|
||||
- Worker eligibility is determined before execution and checked again before dispatch. Direct workers must be idle in orchestrator-only mode; master-participating mode can use reachable compatible peers without requiring an empty queue. Transiently unreachable workers are excluded; healthy candidates can continue. Malformed capability/queue responses and incompatible versions fail explicitly. Workers joining later are not added to an already-running region.
|
||||
- Queue submission checks reachability and feature compatibility without freezing direct-worker selection to that moment's idle snapshot. A selected direct worker busy with an earlier prompt remains a candidate for the queued run. At region start, saved direct candidates are checked again and only idle, compatible workers receive items; transiently unreachable workers are excluded. Malformed capability/queue responses and incompatible versions fail explicitly. Workers excluded by initial reachability or selected later are not added to the queued run, and workers becoming free after region start are not added mid-region.
|
||||
- **Orchestrator Only** can queue while selected direct workers are busy. At region start it requires an idle compatible direct worker or a compatible selected serverless pool; it never falls back to master processing. The queue response's `worker_count` counts saved remote candidates, not guaranteed execution-time participants.
|
||||
- Omitted worker IDs select enabled direct workers and compatible serverless pools. Explicit IDs select the named configured workers/pools, including disabled ones; an empty list stays empty and unknown IDs fail. Pool IDs are retained in Start's hidden selection metadata, not expanded participant IDs. Local direct workers use the master's loopback callback; remote direct workers use its configured public address.
|
||||
- Only the enclosed processing is distributed. List creation and downstream saving run on the master in both modes. With participation enabled, the master is a normal scheduler participant and can run alone when no workers are selected or reachable. **Orchestrator-only mode never silently falls back to master processing.** Failed items are not reassigned automatically.
|
||||
- Local items execute while the enclosing master prompt waits; they are not queued behind that prompt on the master's `/prompt` queue. The enclosed branch is not also evaluated as a full list by the outer prompt.
|
||||
|
||||
+70
-41
@@ -1,12 +1,19 @@
|
||||
# Typed serverless media (PR2)
|
||||
# Serverless images and audio
|
||||
|
||||
`comfy-media-v1` adds typed list results, exact by default, for `IMAGE`, `AUDIO`, `MASK`,
|
||||
`STRING`, and native ComfyUI `VIDEO`. It is an **explicit endpoint profile**, not
|
||||
an automatically detected capability. Omitting `media_profile` still selects
|
||||
`legacy-image-v1`; existing image collectors and handlers keep their old contract.
|
||||
**Distributed Collector** (`DistributedCollector`) collects `IMAGE` and/or `AUDIO`
|
||||
from direct workers and serverless pools. There is no separate
|
||||
`DistributedMediaCollector`; replace that removed node with `DistributedCollector`
|
||||
for image/audio workflows. The Collector has no MASK, STRING, or native VIDEO
|
||||
output sockets.
|
||||
|
||||
`comfy-media-v1` is an **explicit endpoint profile**, not an automatically detected
|
||||
capability. It returns typed image/audio items to the Collector. The standalone
|
||||
bridge's generic contract also supports `MASK`, `STRING`, and native ComfyUI
|
||||
`VIDEO`, but that does not add those outputs to the Collector. Omitting
|
||||
`media_profile` still selects the image-only `legacy-image-v1` contract.
|
||||
See [legacy serverless setup](serverless-workers.md) for credentials and recovery.
|
||||
|
||||
## Use a typed collector
|
||||
## Use the Collector with a typed endpoint
|
||||
|
||||
1. Prepare an endpoint with the [standalone media bridge](../serverless_bridge/README.md)
|
||||
and the workflow's models/custom nodes. The endpoint needs only the bridge,
|
||||
@@ -15,9 +22,9 @@ See [legacy serverless setup](serverless-workers.md) for credentials and recover
|
||||
**Typed media v1 (requires endpoint bridge)** (`media_profile: "comfy-media-v1"`).
|
||||
Start with **Images per run = 1**. This existing setting counts remote jobs,
|
||||
not the number of typed items each job returns.
|
||||
3. Use **Distributed Media Collector** (`DistributedMediaCollector`), connect at
|
||||
least one typed input, and connect its outputs to the appropriate downstream
|
||||
nodes. For literal text, connect a `PrimitiveString` node to `texts`.
|
||||
3. Use **Distributed Collector** (`DistributedCollector`), connect `images`,
|
||||
`audio`, or both, and connect the corresponding outputs downstream. Image-only
|
||||
and audio-only workflows use the same node.
|
||||
4. Queue through Distributed. **Delegate Only** excludes local upstream
|
||||
generation; normal collection includes the master's local values.
|
||||
|
||||
@@ -25,41 +32,55 @@ See [legacy serverless setup](serverless-workers.md) for credentials and recover
|
||||
installation, model availability, or typed execution. Selecting a profile does
|
||||
not install or upgrade anything on the endpoint.
|
||||
|
||||
The collector has five optional inputs and five **list outputs**, in this order:
|
||||
The Collector accepts optional `images` and `audio` inputs, including ComfyUI list
|
||||
inputs, and returns two ordinary outputs—not list outputs:
|
||||
|
||||
| Socket / output index | Native type | Each list item |
|
||||
| Socket / output index | Native type | Aggregation |
|
||||
| --- | --- | --- |
|
||||
| `images` / 0 | `IMAGE` | One BHWC tensor, including its batch dimension |
|
||||
| `audio` / 1 | `AUDIO` | `{ "waveform": BCT_tensor, "sample_rate": positive_int }` |
|
||||
| `masks` / 2 | `MASK` | One BHW tensor |
|
||||
| `texts` / 3 | `STRING` | Exact text, including whitespace or an empty string |
|
||||
| `video` / 4 | `VIDEO` | One supported native video object |
|
||||
| `images` / 0 | `IMAGE` | Concatenate BHWC tensors along the batch dimension |
|
||||
| `audio` / 1 | `AUDIO` | Concatenate waveforms along the sample dimension |
|
||||
|
||||
For each connected socket, order is **master first, then the prepared participant
|
||||
For each connected socket, order is **master first, then prepared participant
|
||||
order, then that participant's item order**—never completion order. Delegate-only
|
||||
mode omits the master. An unconnected socket returns `[]`. A connected remote
|
||||
socket must return 1–64 items; an empty string is an item, an empty list is not.
|
||||
Lists are not concatenated into one tensor, resampled, resized, or aligned across
|
||||
sockets. Different items may have different valid shapes or audio sample rates.
|
||||
The entire remote response is validated and decoded before any of its results
|
||||
are delivered; missing, duplicate, mismatched, or corrupt results fail collection.
|
||||
mode omits the master. Image dimensions/channels and audio batch/channel dimensions
|
||||
must be compatible for concatenation. Use a common audio sample rate; the Collector
|
||||
does not resample or align streams. Audio-only collection returns `None` for images;
|
||||
when no audio is supplied it returns a silent placeholder, not an empty list.
|
||||
|
||||
The typed wire contract carries 1–64 items per connected socket. The entire remote
|
||||
response is validated and decoded before any of its results are delivered;
|
||||
missing, duplicate, mismatched, or corrupt typed results fail collection. Validated
|
||||
wire lists are then combined into the Collector's ordinary outputs.
|
||||
|
||||
Direct workers, `legacy-image-v1` pools and `comfy-media-v1` pools can share an
|
||||
image-only Collector run. Audio requires direct workers or `comfy-media-v1` pools.
|
||||
If any Collector requests `exact`, a selected `legacy-image-v1` pool is rejected
|
||||
before dispatch instead of silently returning 8-bit images.
|
||||
|
||||
## Image return format and quality
|
||||
|
||||
`transfer_format` defaults to **`exact`** and `transfer_quality` to **90**.
|
||||
The Collector's saved `transfer_format` default remains **`legacy_png`**, with
|
||||
`transfer_quality` **90**. For `comfy-media-v1`, the compiler maps `legacy_png` to
|
||||
**`png`**, so normal serverless returns use compressed 8-bit PNG. `exact` is opt-in.
|
||||
These controls affect **remote IMAGE returns only**, not master-local images,
|
||||
uploaded inputs, MASK, AUDIO, STRING or native VIDEO values.
|
||||
uploaded inputs or AUDIO. The bridge's generic MASK, STRING and native VIDEO
|
||||
codecs are also unaffected.
|
||||
|
||||
- `exact`: preserve floating-point dtype and values, with no image quantization.
|
||||
- `exact`: preserve supported floating-point dtype and values, with no image
|
||||
quantization. Available for direct workers and `comfy-media-v1` pools; it is
|
||||
never downgraded to an 8-bit format. An older master that rejects exact transfer
|
||||
causes an error, not a legacy retry. Update both sides before selecting it.
|
||||
- `legacy_png`, `png`, `ffv1`: lossless **8-bit image pixels**, not exact float
|
||||
tensors. Image values are clipped to [0,1] and quantized as in the existing
|
||||
Collector. `legacy_png` uses PNG without compression; `png` compresses it.
|
||||
tensors. For typed serverless returns, image values are clipped to [0,1] and
|
||||
quantized; `legacy_png` is sent as `png`. Direct workers retain the original
|
||||
JSON/base64 framing for `legacy_png`.
|
||||
- `jpeg`, `webp`: lossy still-image compression.
|
||||
- `h264`, `h265`, `av1`: lossy video encoding of each IMAGE tensor's frame batch.
|
||||
This does not transcode the native `video` socket or combine separate list items.
|
||||
- `h264`, `h265` (**experimental**), `av1`: lossy video encoding of each IMAGE
|
||||
tensor's frame batch. This is an image-return encoding, not a native VIDEO output.
|
||||
- Quality 1–100 affects lossy formats only, using the existing Collector's
|
||||
quality-to-CRF mapping for video. Dimensions, frame order and channel count
|
||||
are preserved; alpha is carried separately where needed, not silently dropped.
|
||||
are preserved for typed serverless returns; alpha is carried separately where
|
||||
needed, not silently dropped.
|
||||
|
||||
Both master and endpoint must have this bridge revision for compressed image
|
||||
returns. Pillow handles still formats and CPU/software PyAV encoders handle video;
|
||||
@@ -67,8 +88,11 @@ unavailable codecs fail clearly rather than silently changing the selection.
|
||||
Compression can reduce result egress and inline response size, but does not relax
|
||||
encoded/decoded byte budgets or the input request limit. Compressed IMAGE batches
|
||||
are capped at 240 frames and 8192 pixels per dimension, with a conservative
|
||||
256 MiB aggregate decode budget (including codec scratch and alpha). The original Collector's
|
||||
settings, default and fallback behavior are unchanged.
|
||||
256 MiB aggregate decode budget (including codec scratch and alpha). Direct
|
||||
workers have different limits and non-exact compatibility fallbacks; see
|
||||
[result transfer](compressed-result-transfer.md). The bridge's internal
|
||||
`MediaExportV1` default is still `exact` for generic callers that omit controls;
|
||||
the Collector compiler explicitly supplies `png` for the normal saved default.
|
||||
|
||||
## Inputs and fidelity
|
||||
|
||||
@@ -92,7 +116,11 @@ becoming endpoint-local paths. Models and custom nodes are never transferred.
|
||||
the pure codec itself rejects non-CPU storage. Remote decoding returns
|
||||
independently owned CPU values. STRING transport is exact UTF-8 (`utf8-v1`).
|
||||
|
||||
### Native VIDEO is not a generic video converter
|
||||
### Bridge-level native VIDEO support
|
||||
|
||||
The generic bridge can import/export native VIDEO, and the typed input adapter
|
||||
can stage `LoadVideo` for upstream processing. This is not a Collector VIDEO
|
||||
output or a generic video converter.
|
||||
|
||||
The adapter supports the pinned native classes **exactly**, not arbitrary
|
||||
subclasses or objects exposing `save_to()` / `get_components()`:
|
||||
@@ -127,11 +155,12 @@ invalidate a returned VIDEO.
|
||||
**10,000,000 bytes**. Larger results require an operator-supplied publisher
|
||||
returning expiring public HTTPS artifact URLs. See the bridge guide for
|
||||
bounded download/upload responsibilities and retention.
|
||||
- Remote typed transport is **serverless only**. Direct-worker typed transport
|
||||
is not implemented. Do not mix a typed collector with legacy image collectors,
|
||||
unsupported distributed/batch nodes, or collector-to-collector dependencies.
|
||||
Downstream processing stays on the master. Image-list scheduling is separate
|
||||
PR3 work, not part of this contract.
|
||||
- The generic five-kind typed envelope is a serverless bridge contract. Direct
|
||||
workers support Collector images/audio, including opt-in exact IMAGE transport,
|
||||
not generic MASK/STRING/VIDEO Collector outputs. The typed compiler rejects
|
||||
unsupported distributed/batch nodes and collector-to-collector dependencies.
|
||||
Downstream processing stays on the master. [Image-list scheduling](image-list-queue.md)
|
||||
is a separate execution mode.
|
||||
|
||||
## Timeout is not proof that Comfy stopped
|
||||
|
||||
@@ -154,5 +183,5 @@ the request or turn every handler exception into a safely reusable terminal
|
||||
provider job. Master-side provider status/reconciliation alone is not proof
|
||||
that a surviving local Comfy process or callback has stopped.
|
||||
|
||||
These docs describe the implementation and its offline tests; they do not claim
|
||||
live Runpod/provider or GPU end-to-end validation.
|
||||
These docs describe the implementation, not a test result or live Runpod/provider
|
||||
or GPU end-to-end validation.
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
# Runpod serverless workers (preview)
|
||||
|
||||
A **Serverless (Runpod)** worker is a queue-based endpoint, not a permanent
|
||||
ComfyUI machine. Runpod chooses the actual GPU workers. Distributed submits one
|
||||
normal ComfyUI API workflow per requested image and collects the returned images.
|
||||
The endpoint workers do not need ComfyUI-Distributed installed.
|
||||
ComfyUI machine. Runpod chooses the actual GPU workers. This page describes the
|
||||
default **`legacy-image-v1`** profile: Distributed submits normal ComfyUI API
|
||||
workflows and collects returned image files. For image/audio collection and
|
||||
opt-in exact IMAGE returns, use [the `comfy-media-v1` profile](serverless-media.md)
|
||||
with the same **Distributed Collector** (`DistributedCollector`). The endpoint
|
||||
workers do not need the full ComfyUI-Distributed extension installed.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -14,7 +17,8 @@ The endpoint workers do not need ComfyUI-Distributed installed.
|
||||
4. Save, then choose **Test Saved Endpoint**. This checks authentication and
|
||||
endpoint health only; it does not start GPU inference or prove model/node compatibility.
|
||||
5. Enable the worker when you want queued Distributed workflows to use it.
|
||||
6. Use an image **Distributed Collector** and queue through Distributed as usual.
|
||||
6. Connect the **Distributed Collector**'s `images` input and queue through
|
||||
Distributed as usual. Leave `audio` unconnected for this legacy profile.
|
||||
Enable **Delegate Only** if the master should collect without generating locally.
|
||||
|
||||
Keys are write-only. Leave the key field blank to retain a saved key, or select
|
||||
@@ -152,9 +156,16 @@ Recovery uses the same trusted-master API as the panel; it never resubmits a job
|
||||
Worker setting changes become available once all its local tasks and unresolved records
|
||||
are finished. No credentials or ledger files need to be edited by hand.
|
||||
|
||||
## First-version limits
|
||||
## Legacy profile limits
|
||||
|
||||
- Image collectors only. Audio/video transfer and audio collectors are rejected.
|
||||
- Only the Collector's image branch is supported by `legacy-image-v1`; connected
|
||||
audio is rejected. `comfy-media-v1` adds audio through the same Collector and
|
||||
supports the configured image-return formats.
|
||||
- `exact` is opt-in and requires direct workers or `comfy-media-v1` pools. If any
|
||||
Collector requests it, a selected legacy pool is rejected before dispatch,
|
||||
not used as an 8-bit fallback. Other Collector transfer-format controls do not
|
||||
change this legacy profile's `SaveImage`/image-file contract. With
|
||||
`comfy-media-v1`, the saved `legacy_png` default is mapped to compressed `png`.
|
||||
- Coordinated `UltimateSDUpscaleDistributed` workloads and unsupported Distributed
|
||||
node types are rejected before submission.
|
||||
- Downstream post-collector processing remains local; the remote graph contains
|
||||
@@ -168,6 +179,6 @@ are finished. No credentials or ledger files need to be edited by hand.
|
||||
- No remote GPU discovery, endpoint provisioning, model synchronization, automatic
|
||||
scaling changes, or inbound master callback/tunnel is needed.
|
||||
|
||||
This feature is verified with offline provider/transport tests. A successful
|
||||
health test is not a paid end-to-end generation test; test your deployed handler
|
||||
These docs describe the implementation, not a test result. A successful health
|
||||
check is not a paid end-to-end generation test; validate your deployed handler
|
||||
and model inventory with one image per run before increasing it.
|
||||
|
||||
Reference in New Issue
Block a user