ComfyUI's CLIP.encode_from_tokens() calls load_model() -> model_management.load_models_gpu()
on every encode. With the dynamic VRAM loader (comfy_aimdo, 0.4x+) that re-runs
ModelPatcherDynamic.load() for a model that is already resident: a walk over every module plus
free_memory bookkeeping, visible as one
Model MOSSImageTEModel_ prepared for dynamic VRAM loading. 7002MB Staged. ...
log line per prompt (N prompts -> N+1 lines). USCG's pre-encode paid it once per prompt, and
again on the single-encode paths (model switch, distributed, upscale, Florence2 hires).
Measured on the 5060 Ti (ComfyUI 0.42.5), MOSS-Image TE = Qwen3.5-9B w4a8, same process state:
template path (CLIPTextEncode), per prompt ....... 0.92 s (re-prepare every encode)
USCG batch encode with pinned_clip, per prompt ... 0.84 s (one prepare per pass)
USCG job, 21 prompts + 21 one-step images ........ 36.8 s -> 33.0 s
for scale: SDXL 0.15 s, Krea 2 0.29 s, Qwen-Image 0.32 s per prompt on the template path
The rest is the encoder forward itself: ~0.55 s fixed + ~2 ms/token for Qwen3.5-9B (24 of its
32 layers are Gated DeltaNet run through the pure-torch chunked recurrence, fp32 compute), so
a 60-prompt MOSS grid still spends ~50 s encoding before the first image. That part is
identical to the official workflow.
batch_encoding.pinned_clip(clip, texts): load once through load_model() (sized by the longest
text so any memory estimate still runs), shadow clip.load_model with a no-op for the block,
restore on exit or exception, nesting safe. encode_pair_cached(): positive+negative with one
load, nothing when both are cached. The pre-encode summary line now prints total time and
ms/prompt.
Applied to batch_encode_with_cache, the model-switch pre-encode in the main loop, the
distributed group pre-encode and per-job encode, the deferred-upscale re-encode fallback, the
dashboard upscale runner and the Florence2 hires step.
Tests: tests/test_pinned_clip.py (10) against the real batch_encoding module; the conftest
stub gained pinned_clip/encode_pair_cached. 137 passed, same 15 pre-existing torch-stub failures.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MOSS-Image (Flux.2 latent: 128ch, 16 px per cell) jobs were built with the
SD-era `h // 8`, so every image sampled a latent with 4x the cells and decoded
at 2x the requested size (832x1216 -> 1664x2432): ~10x slower than the official
ComfyUI template (~120 s vs ~10 s per image on the 5060 Ti).
- model_loader: get_latent_downscale_ratio() reads latent_format.spacial_downscale_ratio
(8 for SD/SDXL/Flux.1, 16 for the Flux.2 family); make_empty_latent() builds the
job latent and warns when WxH is not a multiple of the ratio. Both orchestrator
latent sites (main loop + distributed worker) use it; hires-fix latent upscale,
tiled hires sampling and VAE tiling divisors no longer assume 8.
- CLIP type: resolve_clip_type() warns on unknown types instead of silently using
stable_diffusion, and when clip_type is left at the default it follows the
detected model family (MOSSImage -> moss_image, Flux2 -> flux2, ...). MOSS loaded
as a generic Qwen3.5 TE before (wrong chat template). The Builder's CLIP Type
list is now read from ComfyUI's CLIPLoader/DualCLIPLoader (static list = fallback,
now including moss_image); LS_CACHE_VERSION 3 refreshes cached lists.
- Skip Layer Guidance: new Extra Model & Sampling sub-group wrapping
SkipLayerGuidanceDiTSimple (double/single layers, start/end %), plumbed like Deep
Shrink (builder emission, expand_configs, orchestrator, generate_image). Defaults
match the official MOSS-Image workflow (single block 22, 0-50%).
- tests: latent geometry + clip type resolution, dynamic clip type lists, SLG
config emission/expansion (24 new; the 15 pre-existing torch-stub failures unchanged).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Replaces direct imports of distribution_manager/routes with `from . import distribution`.
- All DistributionManager(...) / set_distribution_manager(...) /
notify_workers_to_start(...) / stop_all_workers(...) / _get_master_url()
calls now go through the facade.
- Early-exit guard at distribution branch entry raises clear install
instructions when the companion plugin isn't installed.
- create_manager call passes three callbacks (job_completed_check,
build_prompt_with_triggers, get_model_cache_key) so the companion's
DistributionManager can avoid bidirectional imports back into main USCG.
- _get_master_url() function deleted (moved to companion's usrv_dist_socket).
- _send_distribution_status moved into generation_orchestrator (it was just
a PromptServer.send_sync helper, no network).
The 3 distribution_*.py files in main USCG are now unreferenced. Deletion
happens in subsequent commits.
When Remote URL VAE is configured but the companion plugin
(ComfyUI-USCG-RemoteVAE) isn't installed, the Builder UI shows an install
card and saves an empty URL ('remote:'). extract_remote_vae_url() was
raising a misleading 'URL is empty' message at run time.
Now: if companion is missing -> RuntimeError with install instructions
if companion is installed but URL is empty -> original message
Discovered during Phase C smoke test.
The companion plugin (ComfyUI-USCG-RemoteVAE) resolves endpoint names to
URLs internally. Main USCG no longer needs the URL allowlist - it only
passes the endpoint NAME (e.g. 'SDXL') through the worker.
User report: 'Allocation on device 0 would exceed allowed memory' OOM mid
Florence2 beam search on a 2-image batch with 8 GiB VRAM. Currently allocated
15.5 GiB on what should be a 3-5 GiB workload.
Root cause: dashboard upscale_runner ALREADY loaded meta.model into VRAM as
its session base. My from_manifest path then called load_checkpoint() for
item.model — same checkpoint name, but a SECOND full copy in VRAM (a fresh
ModelPatcher with its own UNet+CLIP+VAE weights). For SDXL that's ~7 GiB
duplicated. Plus the Florence2 model on top.
Fix:
- Pass session_model_name from upscale_runner / orchestrator into Florence2
- _get_or_load_checkpoint_lora short-circuits when item.model == session_model_name:
reuse the session's base handles and just apply the item's LoRAs on top
(load_loras_for_preencoding is a cheap patch, no weight reload)
- Different model name → still does a fresh full load (correct behavior)
- Add comfy.model_management.soft_empty_cache() at end of run_florence2_step
to release intermediate KSampler latents and beam-search buffers before
the next item iterates
96/96 tests still green (the new branch is covered by the existing
test_from_manifest_caches_by_combo + test_from_builder_returns_fallback
cases; same-model short-circuit doesn't change observable behavior for
those mocks).
Ordering bug: my dispatch used existing_data['items'].append(meta) which puts
new entries at the END of the list. The canonical pattern at L2255 uses
insert(0, meta) (newest first, matches flush_batch_with_vae's order). Result:
Florence2 outputs appeared in wrong position relative to other images.
Duplicate base bug: my dispatch never set upscale_produced = True. That flag
gates whether the BASE image gets saved at L2337 — without it, every Florence2
run produced TWO manifest entries (the inpainted result AND the un-inpainted
base), with the same gen_index. Also added rejected:False to match the
canonical entry shape.
UnboundLocalError at runtime: 'cannot access local variable random where it
is not associated with a value'. Cause: Python scopes 'random' as a local
of the entire _run_jobs function (it's imported as a bare name later in the
function at L2471 etc), so any earlier reference triggers UnboundLocalError.
The inline upscale loop imports 'import random as upscale_random' at L1797
specifically to avoid this — my Florence2 code (and seedvr2's at L2035)
used bare 'random.randint' which shadowed the local. Switched both my sites
+ the pre-existing seedvr2 site to upscale_random.
Tests still 96/96 green.
'meta' isn't assigned until L2324, AFTER the inline upscale loop. My
dispatch used dict(meta) twice (sentinel + success), both NameError'd at
runtime with 'cannot access local variable meta where it is not associated
with a value'. Switched both to create_image_metadata(conf, ...) matching
the normal hires/model upscale pattern at L2184.
(The pre-existing seedvr2 inline dispatch at L2025 has the same bug —
out of scope for this fix.)
The inline upscale loop in generation_orchestrator._run_jobs (~L1875) runs
per-job when 'Upscale Pipeline' is enabled in the Builder. It had a seedvr2
elif but no florence2_hires elif — so my mode fell through to combos=[] and
silently did nothing. (The upscale_runner.py path I wired previously only
covers DASHBOARD-triggered upscales, not inline runs.)
Adds the parallel florence2_hires dispatch to the inline path:
- Decodes the just-generated latent to PIL via decode_latent_with_vae
- Forces model_source='from_builder' (the loaded model is what we want for
inline runs; from_manifest would trigger a redundant reload)
- Honors no-detection sentinel + chained-step re-encode flow
- Manifest entry has upscale_source='inline' (vs 'dashboard' for the
dashboard path)
Preflight guard uses a module-global since 'job' doesn't exist in this
scope (each job is just a loop iteration here).
Test count unchanged (96 passing) — this is plumbing-only; unit tests
already cover run_florence2_step + manifest builders. Real-Comfy smoke
test is the verification.
The state_to_configs_json fix (a3ad111) correctly omits the "vae" key
when nothing is selected, but the orchestrator still crashed when a
config carried "vae": "None" from any other path — legacy saved state,
Revise modal edits, distributed worker job specs, etc. Even a fresh
Builder UI node could trigger this if any consumer pushed "None" back
into the entry.
Defense in depth: at all 4 read sites in generation_orchestrator.py,
normalize "None"/"none"/empty/falsy values to "Default" before the
"!= Default" gate that decides whether to load a per-config VAE. With
"Default" the orchestrator falls back to the checkpoint's bundled VAE
instead of calling load_vae_by_name("None") which raises
FileNotFoundError.
Sites normalized: check_if_job_completed VAE comparison (~line 227),
first-conf VAE load (~line 1029, the user's crash trace), per-config
VAE switching in the main loop (~line 1454), distributed-mode VAE
switching (~line 2885).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a new SUB-GROUP 4 in the Builder UI's "Extra Model & Sampling
Options" section with all 7 params from ComfyUI's built-in
PatchModelAddDownscale node:
- block_number (int, default 3)
- downscale_factor (float, default 2.0)
- start_percent / end_percent (float 0-1, default 0.0 / 0.35)
- downscale_after_skip (bool, default true)
- downscale_method / upscale_method (combo: bicubic/nearest-exact/
bilinear/area/bislerp, default bicubic)
Toggle checkbox controls whether the patch is applied. State plumbs:
config_array.use_deep_shrink/* (JS) -> state_to_configs_json -> per-
config entries -> expand_configs base_combos -> generate_image -> looks
up PatchModelAddDownscale via NODE_CLASS_MAPPINGS and calls .patch() to
patch the model before sampling. Logs an "Applied
PatchModelAddDownscale" line on each gen for visibility.
Mirrors the existing model_sampling_override / use_advanced_sampling
patterns for consistency. Hidden on LTX video config arrays since the
patch targets the SDXL/SD UNet.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
User feedback: when 25/26 items already exist and only 1 new is being
generated, dashboard showed "job 14/26" or similar (skip path doesn't
update progress). Now displays effective progress: existing_count +
new_completed / total. So a session with 25 pre-existing shows 25/26
immediately, then 26/26 after the 1 new gen.
- Adds new_completed counter (separate from loop-index current_job)
- Replaces current_job with existing_count+new_completed in dashboard-
facing log/send_sync (lines 2107, 2114, 2123, 2133)
- Sends an initial progress update right after pre-flight so the user
sees the correct "already done" count before the first gen, clearing
any stale values from a prior run
- Other current_job references (LTX dispatch, image-gen log, start_at_job
gate, periodic save, final send, distributed mode) keep iteration-
index semantic — different contexts
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 490c519 added a webp/png/jpg dropdown but only flush_batch_with_vae
honored it. Three other save sites still hardcoded WEBP:
- flush_batch_with_remote_vae (image_generation.py)
- upscale_runner.py (3 sites in inline + deferred upscale paths)
- distribution_worker.py (PIL bytes shipped back to master)
Now all save sites read image_format from session_settings (orchestrator)
or the job spec (distributed) and match the format/extension.
Changes:
- remote_vae.py: RemoteVAEDecodeWorker accepts image_format kwarg; _worker
uses it to pick format/extension at save time (the real save site for the
remote VAE path — flush_batch_with_remote_vae only queues to the worker)
- generation_orchestrator.py: passes image_format to all RemoteVAEDecodeWorker
constructors (initialize_remote_vae + 3 per-config sites); adds image_format
to all 8 existing_data["meta"] writes so upscale_runner can read it from disk
- upscale_runner.py: reads image_format from manifest meta at thread start;
uses _fmt/_ext locals at all 3 save sites (seedvr2, latent-decode, PIL)
- distribution_manager.py: injects _session_image_format into worker job
config alongside existing _session_cooldown passthrough
- distribution_worker.py: reads _session_image_format from config; encodes
bytes in the chosen format; stamps _image_ext on meta so master uses it
- distribution_routes.py: pops _image_ext from meta to build filename with
correct extension before writing bytes to disk
Default behavior unchanged: all paths fall back to webp if image_format
is absent or unrecognised.
Code review found this in 490c519.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a dropdown to the Builder UI's Run Settings section letting users
pick the saved image file format. Default remains webp (current
behavior). State plumbed through state_to_configs_json ->
_session_settings -> orchestrator -> save_image_to_disk.
PNG users will see lossless saves; JPG users smaller files at quality 95.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Scans the planned configs list once before the generation loop and
counts how many already have output files on disk (the same skip path
used by check_if_job_completed). Displays the count in the running
ETA line: "job X/Y (P%) | ETA: ... | N existing".
Helps users gauge how much actual work remains when re-running a
session that's been partially completed in a prior run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds SeedVR2 as a new upscale mode alongside HiRes Fix, Model Upscale,
and Model+HiRes. Requires ComfyUI-SeedVR2_VideoUpscaler to be installed
as a dependency — the plugin calls its nodes programmatically.
Data Model:
- New seedvr2 options object on each upscale step with all SeedVR2
parameters: dit_model, resolution, max_resolution, seed,
color_correction, batch_size, noise scales, blocks_to_swap,
attention_mode, offload_device, cache_model, VAE tiling options
- _defaultSeedVR2Options() / createDefaultSeedVR2Options() helpers
- Migration: ensureStepFields adds seedvr2 defaults to existing configs
Builder UI (conf-builder-config-management.js):
- "SeedVR2 Upscale" added to mode dropdown
- When selected, shows full SeedVR2 settings: DiT model selection
(10 variants: 3B/7B x fp16/fp8/GGUF/sharp), resolution, seed,
color correction, noise scales, batch size, VRAM optimization
(blocks_to_swap, attention_mode, offload), VAE tiling with
conditional encode/decode tile size and overlap fields
Dashboard (logic_upscale_modal.js):
- Same SeedVR2 mode and fields in renderPipelineEditor via _renderStep
- All fields use existing _makeSelect/_makeNumber/_makeCheckbox helpers
Python Execution (image_generation.py):
- New seedvr2_upscale(pil_image, seedvr2_config) function
- Imports SeedVR2 nodes from NODE_CLASS_MAPPINGS at runtime
- Calls SeedVR2LoadDiTModel → SeedVR2LoadVAEModel → SeedVR2VideoUpscaler
- Converts PIL→tensor→SeedVR2→tensor→PIL
- Clear error if SeedVR2 not installed
Pipeline Integration (generation_orchestrator.py):
- Inline upscale path: mode=="seedvr2" branch decodes current latent
to PIL, calls seedvr2_upscale, saves result, updates manifest
- Supports chained steps: non-final SeedVR2 steps re-encode to latent
for the next step in the pipeline
Preset compatible: seedvr2 config serializes naturally as JSON, saved
and loaded with upscale presets without any special handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The master's wait-for-workers loop could block forever if a worker
crashed without cleanup, leaving a job in "Claimed" state. The
_release_timed_out_jobs() was only called during claim_job, never
during the wait loop.
Fixes:
- Call release_timed_out_jobs() every 5s during the wait loop to
reclaim jobs from dead workers (no heartbeat)
- Add configurable wait_timeout (default 600s/10min) from distribution
config — after timeout, force-reclaim all remaining claimed jobs and
proceed to finalization instead of looping forever
- Add public release_timed_out_jobs() method to DistributionManager
- Remaining jobs after timeout are marked completed and abandoned with
a warning, unblocking the ComfyUI queue
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Workers now generate base images only, and the master runs all upscale
pipelines after distributed generation completes. Uses the existing
run_deferred_upscales() function. The warning message now says upscales
will run on master after collection instead of "not supported."
Builds deferred queue from all generated items in existing_data, runs
after remote VAE cleanup but before final manifest save.
Also adds model count endpoint for fast change detection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add _expand_lora_weight_arrays() that expands bracket notation in LoRA
strength fields into Cartesian product of configs:
lora1:[0.5, 1.0]:1.0 + lora2:0.8:[0.7, 1.0] → 4 configs
Distinguishes weight arrays (brackets in strength after :) from folder
random syntax (brackets in name before :). Runs before folder expansion
so both features work together correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The delayed manager cleanup used a flat 30s timer, which could expire
before a slow worker finished submitting its last result. Now polls
every 10s (up to 5 minutes) checking if any registered workers still
have recent heartbeats before cleaning up the manager.
Also added clarifying comment on submit_result route — it intentionally
does NOT check is_active, only manager existence, to accept late
submissions from workers that were mid-generation when master finished.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New 'Run Settings' section in Builder UI with a job number input.
When set > 0, the generation loop skips all jobs before that number.
Useful for resuming at a specific point without relying on
auto-detection. Flows through session settings to the orchestrator.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When run_upscales_at_end is enabled, inline upscale is skipped and
jobs are queued. After generation completes (before cleanup), the
deferred phase loads base images from disk, VAE-encodes them, and
runs full pipeline chains. Includes resume support, interrupt handling,
and progress reporting with phase="upscaling".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace set() with ordered list dedup for batch encoding so prompts
generate in the order they're written in the config array
- Track cumulative upscale duration so the time metric includes both
generation and all upscale steps instead of just the last upscale
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace flat upscale configs with pipeline/steps/repeat architecture:
- Pipelines run independently from base image
- Steps chain sequentially within a pipeline
- Steps support repeat count for iterative refinement
Add "Save Pre-Upscaled Output" option to also keep the base image.
Add "Adjust Prompt During HiRes Fix" with prepend/append/replace
behaviors. HiRes-adjusted prompts are batch pre-encoded for efficiency.
Results written to manifest metadata and exposed as label overlays.
Fix fmtVal scoping error in buildLabelOverlay (was referenced outside
the upscale block where it was defined).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three bugs fixed:
1. Upscaled images used bare 'filename=...' instead of full '/view?filename=...&type=output&subfolder=...' URL format, causing 404s in dashboard
2. Upscaled manifest entries missing 'id' and 'rejected' fields, causing holes in dashboard grid
3. Base (non-upscaled) images were still saved to pending_batch when upscaling was enabled — now only upscaled final versions appear in output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Skip upscale combos that require a model when none is selected.
Use effective_size=2.0 for model_only mode instead of 1.0 which
would resize the model output back to original resolution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Seed: expanded per-gen behavior + new full run seed behavior (random/increment/decrement before/after entire run)
- Canvas: fix flicker during simultaneous pan+zoom via requestAnimationFrame batching
- Upscaling: 3 session-level modes (HiRes fix, model upscale, model+HiRes) with tiled VAE support
- Cooldown: configurable GPU breaks between generations with optional VRAM clearing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, check_if_job_completed() with optional inputs would only
match on sampler/scheduler/steps/cfg/denoise/seed — skipping model,
prompt, and lora checks. This caused false positive matches, skipping
jobs even when the user changed models via optional input nodes.
Now returns -1 (no match) immediately when has_optional_inputs=True,
forcing all jobs to re-generate since upstream changes can't be detected.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The conditioning serialization was only preserving pooled_output, dropping
critical keys like attention_mask that Wan/T5 models need for proper
sampling. This caused KSampler to produce NaN latents on distributed workers.
Includes backwards compatibility with old serialization format.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ComfyUI v16.3's Dynamic VRAM uses ModelPatcherDynamic which asserts
force_patch_weights=False. Removed the flag from both CLIP GPU loading
calls to match ComfyUI's own CLIP.load_model() pattern. Backwards
compatible with older ComfyUI versions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix Python 3.10 SyntaxError: extract backslash replace() out of f-string
expression in config_builder_node.py (issue #12)
- Fix dashboard iframe stealing focus from ComfyUI on every live update
by removing viewport.focus() from render/update paths in logic_virtual.js
- Intercept Ctrl+S in dashboard iframe and forward to ComfyUI parent
window so workflow save works instead of triggering browser Save As dialog
- Config Builder VAE settings now override sampler node's remote_vae_endpoint
via config_overrides_vae flag in generation_orchestrator.py flush logic
- Fix Shift+1-9 column hotkeys: use e.code instead of e.key since Shift
produces symbols (!@#$) not digits for e.key on standard keyboards
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The DistributionManager, config builder, and UI all already used 600 as
the default, but the orchestrator's fallback was still 300. This caused
premature job reclaim when workers loaded models for the first time.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Worker downloads missing models from master via HTTP before processing.
Background pre-fetch thread downloads upcoming models during job execution.
Distribution manager scans pending queue for unique model references.
Job claim responses include upcoming_models list when sync enabled.
Master passes sync_models_to_workers flag to workers via start_worker.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each config array can now have its own resolution list displayed as chips with
categorized preset dropdown (SD 1.5/SDXL/Flux × Square/Portrait/Landscape) plus
custom W×H input. Resolutions flow through configs_json, expand in the Cartesian
product, and override the sampler's global resolutions_json on a per-config basis.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
