FearL0rd
5fca7fa289
Critical memory optimization for Parallel Anything node to prevent CUDA out-of-memory errors on first execution and improve stability across multi-GPU setups. Problem: - First-run OOM errors occurred due to CUDA memory fragmentation and holding duplicate model copies (CPU + GPU) simultaneously during cloning - No handling for devices that failed during cloning due to insufficient VRAM - Original model restoration caused temporary 2x VRAM usage spikes Solution: - Skip cloning when target device matches source device (use reference) - Incremental state_dict loading to minimize peak memory usage - Aggressive ComfyUI model cache unloading before cloning operations - Progressive cleanup between device clones with synchronized CUDA operations - Graceful degradation: skip OOM devices and redistribute workload Technical Changes: - Added aggressive_cleanup() helper for forced GC and CUDA cache clearing - Modified safe_model_clone() with incremental parameter loading and early-exit if model already on target device - Updated setup_parallel() to use reference counting for original device - Implemented successful_devices tracking for OOM fallback handling - Added periodic torch.cuda.empty_cache() during large model transfers - Enhanced error handling with device-level try/except blocks - Fixed state dict parsing for nested modules in incremental loading Memory optimizations: - Unload ComfyUI models (unload_all_models) before cloning - Delete state_dict entries immediately after transfer - Force CPU transition only when necessary (not when reusing original) - Synchronize CUDA devices before/after memory operations Breaking Changes: None Backwards Compatible: Yes
15 lines
1.0 KiB
TOML
15 lines
1.0 KiB
TOML
# pyproject.toml
|
|
[project]
|
|
name = "ComfyUI-ParallelAnything" # Unique identifier for your node. Immutable after creation.
|
|
description = "This suite of nodes unlocks high-performance parallel processing in ComfyUI by utilizing **Model Replication**. Unlike standard offloading which moves a single model instance between devices, these nodes create independent replicas of the model on each selected GPU/CPU, allowing for true simultaneous batch processing."
|
|
version = "1.0.5" # Custom Node version. Must be semantically versioned.
|
|
license = { file = "LICENSE.txt" }
|
|
dependencies = [] # Filled in from requirements.txt
|
|
|
|
[project.urls]
|
|
Repository = "https://github.com/FearL0rd/ComfyUI-ParallelAnything"
|
|
|
|
[tool.comfy]
|
|
PublisherId = "fearl0rd" # TODO (fill in Publisher ID from Comfy Registry Website).
|
|
DisplayName = "ComfyUI-ParallelAnything" # Display name for the Custom Node. Can be changed later.
|
|
Icon = "https://raw.githubusercontent.com/FearL0rd/ComfyUI-ParallelAnything/refs/heads/main/icon.png" # SVG, PNG, JPG or GIF (MAX. 800x400px) |