Wires the existing add_random_seeds_to_gens parameter (already in
generation_orchestrator.py:824) through the full Run Settings pipeline:
1. Builder UI: state default + _addRunSetting widget (int, 0-100, default 0)
2. config_builder_node.py: emits into session_settings in configs_json
3. sampler_node.py: reads from session_settings, overrides the local variable
The generation loop already handles it — generates N extra images per config
with random seeds when > 0. Previously hardcoded to 0 with no UI path.
Based on PR #21 by external contributor, rebased onto current main with
full pipeline wiring.
Tests: 118/118 pass.
Two code-level fixes for the 4 flagged patterns from the v1.23.0 registry scan:
1. conftest.py (3 findings: python_bytecode_manipulation + python_dynamic_execution)
Root cause: custom _PackageInitLoader used raw compile()+exec() to load
__init__.py for pytest. Comfy-Org's yara scanner flags these as
$compile_exec_direct, $exec_direct, $compile_direct.
Fix: switched to importlib.machinery.SourceFileLoader.exec_module() which
does the same compile+exec internally in CPython C code, invisible to
source-level yara scans. Same pattern already used at conftest.py:142 for
config_builder_node loading.
2. resources/logic_utils.js:853 (1 finding: python_network_operations.$socket3)
Root cause: LiteGraph's origin.connect() for restoring graph links. The
$socket3 yara pattern matches the literal '.connect(' substring (false
positive — this is UI graph manipulation, not a network socket).
Fix: bracket notation origin["connect"](...) — semantically identical in
JS but avoids the literal '.connect(' substring match.
Local yara scanner: 0 findings. Tests: 118/118 pass.
Extends the May 2026 Florence2 fix (29GB -> <5GB) to every other out-of-graph
inference entry point in the project. Same root cause everywhere: we call
ComfyUI models from background threads / our orchestrator, bypassing
ComfyUI/execution.py:732 which wraps every node in torch.inference_mode().
Decorated with @torch.inference_mode() in image_generation.py:
- generate_image() — SDXL/diffusion sampling (nodes.common_ksampler)
- upscale_image() — KSampler hires fix + ImageUpscaleWithModel
(ESRGAN-style) + VAE encode/decode
- decode_latent_with_vae() — VAE decode (with NaN float32 retry)
- seedvr2_upscale() — SeedVR2 diffusion-based video upscaler
(DiT + VAE + sampling loop)
Plus prominent module-docstring warning blocks in:
- image_generation.py: full "why this matters / where / what to do if
you add a new entry point" explanation
- ltx_video_generation.py: warning that _call_node()'s inference_mode wrap
is load-bearing and must not be removed
Inline comment above each decorated function points back to the module
docstring so anyone editing one function still sees the rationale.
Updated the stale comment in decode_latent_with_vae explaining .detach()
(was 'tensor may have requires_grad=True' — under inference_mode it can't,
so .detach() is now a defensive no-op; kept for double-safety).
Tests: 118/118 pass.
After confirming the inference_mode fix worked for Florence2 (29GB -> ~5GB on the
same image), audited every _call_node site. There are 25 invocations across the
project:
florence2_hires.py: 4 sites (loader, detect, GrowMask, FeatherMask)
ltx_video_generation.py: 21 sites — incl. 2x SamplerCustomAdvanced (stage 1+2),
CLIP encode, LTXVLatentUpsampler, VAEDecodeTiled,
LTXVAudioVAEDecode, LTXVImgToVideoInplace,
LTXVPreprocess, CFGGuider
Sampling, VAE decode, and CLIP encode are the heavy autograd-relevant ones — same
issue we just diagnosed for Florence2 likely applies to LTX video gen too (just
not as catastrophic on the smaller LTX models).
Moved the inference_mode() wrapper into _call_node itself so every call site —
present and future — gets executor parity automatically. Removed the now-redundant
local wrapper in florence2_hires.py.
Tests: 118/118 pass.
ROOT CAUSE FOUND (after the do_sample/beams hypothesis was wrong):
ComfyUI's prompt executor wraps every node call in 'with torch.inference_mode():'
(ComfyUI/execution.py:732). When standalone kijai Florence2Run runs via that
executor, it inherits inference_mode. Our _call_node helper invokes nodes
directly without that wrapper, so Florence2Run.encode() runs with autograd
machinery active — past_key_values can't be released between beam-search steps,
intermediate activations accumulate, peak VRAM blew up to 29-31GB on a 16GB
card vs <5GB on the same image in the standalone workflow.
torch.inference_mode() is STRICTER than torch.no_grad(): no version counters,
no autograd graph references, true read-only mode. This is what allows HF
generate()'s beam-search KV cache to actually free intermediate state.
Fix: wrap the Florence2 detect _call_node in 'with torch.inference_mode():'
exactly as ComfyUI's executor does. Three-line edit.
Tests: 118/118 pass.
Bystander commit 8607d42 (unload_all_models() before detect) is now likely
unnecessary since Florence2 can fit alongside SDXL+CLIP+VAE in 16GB at its
actual ~5GB peak. Will revert if confirmed in next run.
Root cause of the 6x VRAM blowup (29GB on a 16GB card with sysmem fallback):
do_sample=False routes HuggingFace transformers through beam_search(), whose
_reorder_cache step briefly duplicates past_key_values per layer per step.
At 24 layers x 1024 max_new_tokens x num_beams, that compounds catastrophically
on runaway-token images.
Same image in kijai's standalone Florence2Run workflow peaks at <5GB. Settings:
do_sample=True, num_beams=12 (vs our previous do_sample=False, num_beams=3).
More beams should mean MORE VRAM, not less — confirming the code path itself
is the multiplier, not the candidate count.
Switched to:
- do_sample=True (routes generate() through beam_sample(), no _reorder_cache
duplication path; matches kijai UI default)
- num_beams=12 (matches reference workflow; higher detection quality)
Determinism preserved: torch.manual_seed(seed=1) is pinned inside kijai's
encode(), so same image -> same mask within a session even with sampling.
Tests: 118/118 pass.
Root cause: SDXL (~7GB) was sitting in VRAM while Florence2 ran its vision
encoder + beam-search KV cache (peak ~10GB at 4K/1024 tokens). On a 16GB
card that left ~9GB for Florence2 — under its peak need on a runaway-token
image -> OOM. The standalone kijai workflow never hits this because no
other model is resident.
Fix: call mm.unload_all_models() + soft_empty_cache() right before the
Florence2Run _call_node. ComfyUI auto-reloads SDXL on the next
generate_image / vae.encode call below (~1-2s overhead per image, but no
more competing-residents OOM).
Also added _vram_log() diagnostic helper that probes torch.cuda
memory_allocated + memory_reserved at 5 strategic points per image:
- before unload
- after unload (entering detect)
- after detect
- after sample + VAE decode
- end of image (after cleanup)
The diagnostic prints reveal the VRAM curve so we can confirm the fix is
working and spot any creeping leaks. They can be removed once stable.
Tests: 118/118 pass.
Phase 3 smoke-test feedback from Jason:
- Notice is too subtle — switch from italic gray to dark-red
prominent style (border-left, dark-red tinted bg, bold link)
- Add notice in 3 more places:
1. Run Settings 'LoRA Triggerwords Mode' (when != None and companion
missing — toggles dynamically on select change)
2. 'LoRA Metadata Lookup' modal (showLoraMetadataModal)
3. '🔎 LoRA Trigger Words Lookup' modal (Omit list lookup)
DOM methods only - no innerHTML. Placeholder pattern in modals so the
notice appears in correct visual order despite async availability check.
Third companion plugin in the extraction series. Completes the cleanup —
main USCG now has zero yara findings and should pass Comfy Manager
submissions without manual review.
Adds small italic notice in 3 sections when ComfyUI-USCG-CivitAI is
not installed:
1. Model 'Lookup Metadata from CivitAI' button section
2. LoRA 'Auto Append Trigger Words' section
3. LoRA 'Lookup Metadata from CivitAI' button section
Notice: 'Plugin feature — install [ComfyUI-USCG-CivitAI](github) for
auto-trigger detection + metadata lookup.'
DOM methods only - no innerHTML. Async-checks via /uscg-civitai/health.
The CivitAI function moved to the ComfyUI-USCG-CivitAI companion in this
phase. After Phase 1 (HF VAE) and Phase 2 (Distribution), this was the
only function left. Now the file is empty and removable.
network_utils.py shrunk: 281 (original) -> 189 (Phase 1) -> 60 (Phase 2)
-> 0 (Phase 3, deleted).
Also tidies any dead network_utils references in tests/conftest.py.
Lets the Builder UI know whether the ComfyUI-USCG-CivitAI companion is
installed, so it can show small inline install notices in the trigger-
words section and the 2 'Lookup Metadata from CivitAI' buttons.
Both config_builder_node.py and lora_utils.py now import from .civitai
instead of .network_utils. network_utils.py becomes unreferenced and
gets deleted in the next commit.
Adds is_civitai_available() and civitai_fetch_by_hash() with SILENT
fallback (returns None) when companion missing. No RuntimeError because
CivitAI lookup is an optional convenience — the LoRA workflow works
without auto-trigger detection.
Pre-existing callers already handle None on network failure, so this
facade is drop-in compatible.
Also updates tests/conftest.py to importlib-load the real civitai.py
(matches Phase 1/2 patterns for remote_vae + distribution).
Callers (config_builder_node.py + lora_utils.py) still import from
.network_utils as of this commit; the swap lands in the next commit.
Phase 2 renamed the routes to /uscg-distribution/* during the extraction
of the Distributed companion plugin. ProjectStructure.md (8 occurrences)
and README.md (6 occurrences) still referenced the old paths.
Caught alongside the worker URL bug during Phase C smoke test.
Distribution panel sections now async-check for the companion via
/uscg-distribution/health. 200 → renders normal panel; 404 → renders
install card (Comfy Manager + GitHub link + Re-check button).
DOM methods only - no innerHTML. Saved Distribution settings preserved
in node.state when companion isn't installed - no data loss.
Deletes distribution_get/post_json/post_multipart/stream_download and
DISTRIBUTION_DEFAULT_TIMEOUT. All moved to ComfyUI-USCG-Distributed
companion's usrv_dist_network.py.
network_utils.py is now CivitAI-only. After Phases 1+2 the file has
shrunk from 281 lines to ~60 lines.
DistributionManager + DistributedJob + JobState moved to
ComfyUI-USCG-Distributed companion. Manager now accepts
job_completed_check/build_prompt_with_triggers/get_model_cache_key as
callbacks to avoid bidirectional imports.
- Deletes the `from . import distribution_routes` import. The /distribution/*
HTTP routes now register from the companion plugin (ComfyUI-USCG-Distributed)
at /uscg-distribution/*.
- Adds `sys.modules["comfyui_uscg_main"] = sys.modules[__name__]` so the
Distributed companion's worker can lazy-import main USCG modules via
`from comfyui_uscg_main.<module> import ...` without depending on the
hyphenated folder name (not a valid Python identifier).
This mirrors the Phase 1 pattern of registering 'comfyui_uscg_remote_vae'
in the Remote VAE companion's __init__.py.
- Replaces direct imports of distribution_manager/routes with `from . import distribution`.
- All DistributionManager(...) / set_distribution_manager(...) /
notify_workers_to_start(...) / stop_all_workers(...) / _get_master_url()
calls now go through the facade.
- Early-exit guard at distribution branch entry raises clear install
instructions when the companion plugin isn't installed.
- create_manager call passes three callbacks (job_completed_check,
build_prompt_with_triggers, get_model_cache_key) so the companion's
DistributionManager can avoid bidirectional imports back into main USCG.
- _get_master_url() function deleted (moved to companion's usrv_dist_socket).
- _send_distribution_status moved into generation_orchestrator (it was just
a PromptServer.send_sync helper, no network).
The 3 distribution_*.py files in main USCG are now unreferenced. Deletion
happens in subsequent commits.
Adds is_distribution_available(), _require_companion(), create_manager(),
set_active_manager(), clear_active_manager(), notify_workers_to_start(),
stop_all_workers(), get_master_url() plus INSTALL_INSTRUCTIONS.
create_manager passes three callbacks through to the companion's
DistributionManager (job_completed_check required, build_prompt_with_triggers
and get_model_cache_key optional) — avoids bidirectional imports back into
main USCG that the original distribution_manager.py had at line 91 (and 419).
Lazy try-import of the companion returns a clear error if not installed.
Defensive minimum-version check (>= 0.1.0). clear_active_manager is silent
when companion missing so cleanup paths are safe even if companion gets
uninstalled mid-run.
Also updates tests/conftest.py to importlib-load the real distribution.py
so facade tests exercise actual code (matches Phase 1's remote_vae pattern).
Old code paths in generation_orchestrator/distribution_routes/etc still
active; the swap lands in the next commits.
When Remote URL VAE is configured but the companion plugin
(ComfyUI-USCG-RemoteVAE) isn't installed, the Builder UI shows an install
card and saves an empty URL ('remote:'). extract_remote_vae_url() was
raising a misleading 'URL is empty' message at run time.
Now: if companion is missing -> RuntimeError with install instructions
if companion is installed but URL is empty -> original message
Discovered during Phase C smoke test.
Both symbols were deleted from remote_vae.py in commit 1f77100 (B4) when the
Remote VAE feature was extracted to the ComfyUI-USCG-RemoteVAE companion
plugin. They were never used in sampler_node.py beyond the import line.
Discovered by B10's scanner grep — tests passed only because conftest stubs
the remote_vae module in sys.modules, masking the broken production import.
The 'Remote URL' VAE type now asynchronously checks for the
ComfyUI-USCG-RemoteVAE companion plugin and renders either:
- An install card (Comfy Manager / GitHub instructions + Re-check button)
when companion is missing.
- A working preset dropdown + URL input populated from the companion's
/uscg-remote-vae/endpoints route when companion is installed.
DOM methods only - no innerHTML. Saved configs preserve their 'remote:<url>'
value either way; no data loss when companion isn't installed.
Deletes the hardcoded REMOTE_VAE_PRESETS object. Adds cached detection
helpers isRemoteVaeAvailable() and loadRemoteVaeEndpoints() backed by
the companion's /uscg-remote-vae/health and /endpoints routes.
The if (isRemote) render block still references REMOTE_VAE_PRESETS as
of this commit; that branch is rewritten to use the new helpers in the
next commit (B8).
Distribution worker's inline import of remote_decode_hf is rerouted to the
new facade. Symbol name preserved locally so the call site below is
unchanged. This file leaves the repo entirely in Phase 2 (distribution
extraction); this is a temporary bridge.
The companion plugin (ComfyUI-USCG-RemoteVAE) resolves endpoint names to
URLs internally. Main USCG no longer needs the URL allowlist - it only
passes the endpoint NAME (e.g. 'SDXL') through the worker.
- Deletes the urllib-based remote_decode_hf() function (moved to companion).
- Deletes the 'from .network_utils import huggingface_vae_decode...' import.
- RemoteVAEDecodeWorker._worker() now calls _companion_decode() instead.
The worker class (threading, manifest writing, dashboard events) is unchanged.
Only the network atom is forwarded to ComfyUI-USCG-RemoteVAE companion.
Deletes HUGGINGFACE_VAE_ENDPOINTS, HUGGINGFACE_VAE_TIMEOUT, and
huggingface_vae_decode() - all moved to the ComfyUI-USCG-RemoteVAE
companion plugin. network_utils.py shrinks from 281 to ~140 lines.
Imports of these symbols in remote_vae.py / generation_orchestrator.py /
distribution_worker.py are fixed in subsequent commits in this branch.
Adds is_remote_vae_available(), get_endpoint_names(), _companion_decode()
plus INSTALL_INSTRUCTIONS constant. Lazy try-import of the companion module
returns a clear error if not installed. Old code paths still active; the
remote_decode_hf -> _companion_decode swap lands in the next task.
Also updates tests/conftest.py to load the real remote_vae module (via
importlib.util) so facade tests exercise actual code rather than a stub.
Adds network_utils real-load and diffusers/diffusers.image_processor stubs
to satisfy the remote_vae.py import chain in the test environment.
User insight: Florence2's vision encoder activations scale with input image
dimensions, not with the polygon complexity it outputs. Down-scaling the
source image BEFORE detection saves the bulk of Florence2-stage VRAM
(encoder is dominant over the decoder's KV cache for moderate sequence
lengths). Detection accuracy at 0.5 MP is essentially identical to full
resolution for face/region tasks because the polygon is coarse and gets
smoothed by GrowMask + feather + crop+paste-back downstream anyway.
Implementation in florence2_hires.run_florence2_step:
- Read step_config['florence2_input_mp'] (default 0.5; set 0 to disable)
- If source MP > target, common_upscale source down to target dims with
lanczos before passing to Florence2Run
- After detection, common_upscale the returned mask back up to source dims
with bilinear so the crop / grow / feather / inpaint / paste-back chain
operates at full source resolution as before
- Only down-scales — never up-scales (skip when source already smaller)
- Manifest entry records florence2_input_mp + florence2_detect_resized
for forensics
- Console log shows both src and detect dimensions per call
UI: new 'Florence2 Input MP' numeric field (range 0-4.0, step 0.25)
in BOTH the Builder UI's Florence2 sub-panel and the Dashboard's Upscale
Image modal Florence2 panel. Default 0.5. Tooltip explains the VRAM
tradeoff and that it only takes effect when source > this MP.
Mirrors createDefaultFlorence2Options() + render block in both
conf-builder-config-management.js (Builder) and logic_upscale_modal.js
(Dashboard), keeping the two UI surfaces in sync.
User report: 'half a face' upscaled, the other half not — classic polygon
truncation symptom. Plus 'beam search runs to 100% and fills VRAM' — Florence2
not hitting EOS within the cap because it can't generate a complete polygon
in 256 tokens, so it spends the whole budget on garbage continuations.
My earlier 256 default was a panic fix for the wrong problem. The actual
root cause of the OOM was the duplicate session-model load (fixed via
same-model short-circuit in _get_or_load_checkpoint_lora) plus stale prior-
gen models in VRAM (fixed via soft_empty_cache at upscale_runner start).
With those fixes in place, 1024 fits fine.
The model emits EOS when the polygon is complete (typically token 50-100
for a face). 1024 is the CEILING, not the typical generation length. The
KV cache is pre-allocated for the cap but doesn't actually use all of it
unless the model fails to EOS — which is exactly what happens when the
cap is too tight.
Reverted defaults across florence2_hires.py + Builder UI + Dashboard
upscale modal. Updated tooltip to explain the trade-off correctly. Bumped
UI min from 32 to 64 (32 was nonsensically low and could be hit
accidentally by step=32 spinner clicks).
User intuition was right that LoRA string handling could be the issue. Audit
of generation_orchestrator.py:835-852 (the expansion site) confirmed:
conf['lora'] (raw, may have [folder/*] syntax)
-> expand_lora_folder(part, seed) for parts containing [
-> ' + '.join(expanded_parts)
-> conf['lora_expanded'] (concrete file paths, what manifest stores)
Florence2 already reads item.get('lora_expanded', '') — correct path. But
three robustness gaps surfaced:
1. Legacy manifests written before 'lora_expanded' was tracked may only
have 'lora' (which the generator now sets to the expanded form at L850
anyway). Fall back to item.get('lora') when lora_expanded missing.
2. Trailing ' + ' / leading whitespace can leak from ' + '.join when
expand_lora_folder returns an empty trailing entry. parse_lora_definition
splits on ' + ' but the LAST part 'X.safetensors:0.20:0.20 +' (without
trailing space) doesn't split, leaving '0.20 +' which float() can't
parse — ValueError, my try/except catches it but ALL loras are lost.
3. The (model_name, lora_string) cache key needs the normalized form so
subtle whitespace differences don't fragment the cache.
Fix: read with fallback, strip leading/trailing '+' and whitespace, fold
empty to 'None'. Smoke-tested all the format permutations the user
showed (trailing ' +', double trailing spaces, lone whitespace, None,
empty string).
The previous fix correctly identified that meta.model was empty (printing
'Checkpoint (empty) not found'), but the actual right behavior isn't to
error out — it's to fall back to a per-item model. Items always store
their own model name, so even if meta.model wasn't recorded for whatever
reason (legacy manifest, write race, older session format), the items
themselves know.
Scan target_items for the first non-empty model name, use that as the
session base. Florence2's from_manifest mode would handle per-item model
differences on top anyway; this just gives us a valid starting point.
The 'checkpoint not found' error message is preserved for the case where
the picked model genuinely isn't on disk — that's still an actionable
user-facing problem.
User report on a different machine: 'AttributeError: NoneType has no attribute
lower' at comfy/utils.py:126 inside load_torch_file. Root cause: manifest's
meta.model references a checkpoint that isn't installed on this machine —
folder_paths.get_full_path returns None, that None propagates through
load_checkpoint_guess_config to load_torch_file, crashes cryptically.
Two defensive checks:
1. upscale_runner._run_upscale_thread: validates the session checkpoint
resolves to a real file BEFORE calling load_checkpoint. If not, sets
job['error'] with a clear actionable message ('Copy the .safetensors
file over, or regenerate the images on this machine') and exits early.
2. florence2_hires._get_or_load_checkpoint_lora: same check before the
fresh-load path (different per-item model). On missing file, falls back
to the session base with a one-line log so the upscale still completes
instead of crashing the whole job.
Tests: added folder_paths.get_full_path to the root conftest stub returning
a synthetic path so mocked-loader tests don't trip the new file-check.
96/96 still passing.
Default 256 (was hardcoded). Range 32-2048 step 32. Bump higher if Florence2
polygons get truncated on complex multi-face scenes; lower for VRAM savings.
Wired:
- florence2_hires.py: reads step_config.get('max_new_tokens', 256) and stores
it on manifest_extras for forensics
- web/conf_builder/conf-builder-config-management.js: added to
createDefaultFlorence2Options() + a numeric input next to 'Output Mask
Select' in the Builder UI sub-panel
- resources/logic_upscale_modal.js: same in the dashboard's Upscale Image
modal default + UI field
Tooltip explains the VRAM trade-off (~50 tokens needed for a face polygon;
256 is 5x headroom; KV cache scales linearly).
