Add explicit dtype alignment before matrix multiplication in _rgb_to_lab_batch
and _lab_to_rgb_batch to prevent float64 promotion from torch.pow operations.
Fixes RuntimeError: expected mat1 and mat2 to have the same dtype.
Improves PR #428 by checking actual VAE device (self.device.type == 'mps')
instead of system MPS availability. Only clears cache when VAE operations
are running on MPS, avoiding unnecessary overhead when VAE runs on CPU.
This fix addresses the PyTorch MPS memory leak (pytorch/pytorch#155060)
where padding operations in convolutions accumulate memory during encode
and decode. Clears MPS cache after each operation to prevent accumulation.
Fixes#363 (absurd MPS VRAM usage - 52GB → 12GB)
Fixes#410 (macOS system restarts from memory exhaustion)
Fixes#415 (inability to upscale beyond 2K resolution)
May help #417 (convolution errors under memory pressure)
Building on @thehhmdb's fix in PR #418 which identified the stderr
pipe blocking issue. This refinement simplifies the solution by
redirecting stderr to DEVNULL and adding stdin.flush() to prevent
buffering deadlocks.
Improvements:
- Simpler implementation without threading complexity
- Zero memory overhead
- Better error messages for debugging
- Maintains fix for the original hanging issue
Co-authored-by: thehhmdb <thehhmdb@users.noreply.github.com>
Fixesnumz/ComfyUI-SeedVR2_VideoUpscaler#418
- Implement node registration in `__init__.py` to fix "Node does not exist" error.
- Implement automatic fallback from `inductor` to `cudagraphs` in `torch.compile` when Triton is missing (Windows fix).
- Optimize VAE `InflatedCausalConv3d` to use 2D convolution path for spatial-only operations, improving speed.
- Optimize VAE `ResnetBlock3D` to support flattened 4D execution path to reduce reshape overhead.
- Update `VideoAutoencoderKL` to default `force_upcast=False` for FP16 inference.
- Add `conv2d`/`conv3d` dequantization support to `GGUFTensor` for GGUF model compatibility.
- Implemented "4D execution mode" in ResnetBlock3D to flatten temporal dimension when convolutions are effectively 2D, reducing reshape overhead.
- Updated InflatedCausalConv3d to support direct 4D input processing and use 2D convolution optimization path.
- Enhanced InflatedCausalConv3d check_effective_2d to strictly verify stride/dilation/padding compatibility.
- Fixed 4D input handling in InflatedCausalConv3d forward pass.
- Implemented 2D convolution optimization in `InflatedCausalConv3d` to speed up spatial-only operations by using `F.conv2d` instead of `Conv3d`.
- Added `torch.nn.functional.conv2d` and `torch.nn.functional.conv3d` to `GGUFTensor`'s `__torch_function__` dispatch to enable automatic dequantization of weights, supporting GGUF models.
- Updated `src/models/video_vae_v3/modules/attn_video_vae.py` to default `force_upcast` to `False` for better FP16 performance.
- Fixed a bug in VAE slicing logic where `slicing_latent_min_size` could become 0, now clamping it to 1.
- Updated `Upsample3D` and `Encoder3D` in `attn_video_vae.py` to use `init_causal_conv3d` for 1x1x1 convolutions, enabling the 2D optimization path.
- Implemented 2D convolution optimization in `InflatedCausalConv3d` to speed up spatial-only operations by using `F.conv2d` instead of `Conv3d`.
- Added `torch.nn.functional.conv2d` and `torch.nn.functional.conv3d` to `GGUFTensor`'s `__torch_function__` dispatch to enable automatic dequantization of weights, supporting GGUF models.
- Updated `src/models/video_vae_v3/modules/attn_video_vae.py` to default `force_upcast` to `False` for better FP16 performance.
- Fixed a bug in VAE slicing logic where `slicing_latent_min_size` could become 0, now clamping it to 1.
- Implemented 2D convolution optimization in `InflatedCausalConv3d` to speed up spatial-only operations by reshaping effectively 2D tensors and using `F.conv2d` instead of `Conv3d`.
- Added `torch.nn.functional.conv2d` and `torch.nn.functional.conv3d` to `GGUFTensor`'s `__torch_function__` dispatch to enable automatic dequantization of weights, allowing the VAE to utilize GGUF quantization.
- Fixed a bug in `VideoAutoencoderKL` where `slicing_latent_min_size` could become 0 with small split sizes, now clamping it to a minimum of 1.
- Replace tensor.scatter_() with torch.argsort() + torch.index_select(out=)
- Uses fundamental PyTorch ops for improved reliability across CUDA/ROCm/MPS
- Aggressive early tensor deletion to minimize memory overhead
- Affects _histogram_matching_channel and _histogram_match_1d in color_fix.py
Related: #351
Move ffmpeg check from FFMPEGVideoWriter to argument validation phase.
Prevents wasted GPU processing time when ffmpeg backend is selected
but ffmpeg is not installed.
- Use try/except instead of blanket MPS check for bicubic+antialias
- PyTorch 2.8.0+ MPS: native fast path (no overhead)
- PyTorch < 2.8.0 MPS: CPU fallback on NotImplementedError
- Add CPU fallback for F.interpolate with antialias=True on MPS (aten::_upsample_bicubic2d_aa not implemented)
- Revert torch.mps.synchronize() calls introduced in v2.5.21 for consistent behavior with CUDA pipeline
- Add --video_backend flag: 'opencv' (default) or 'ffmpeg'
- Add --10bit flag: enables x265/yuv420p10le for reduced banding
- Without --10bit, ffmpeg uses x264/yuv420p for max compatibility
- FFMPEGVideoWriter class with cv2.VideoWriter-compatible interface
- Validates ffmpeg availability before encoding
Based on PR #409 by thehhmdb
- Skip CPU tensor offload on MPS (no memory benefit, causes sync stall)
- Keep input_images and final_video on MPS device
- Add explicit MPS sync at phase boundaries for accurate timing
- Preload text embeddings before Phase 1 to avoid Phase 2 stall
- Skip model→CPU movement before deletion on MPS cleanup
Add ensure_bitsandbytes_safe() shim to handle broken/partial bitsandbytes
installations that cause PyTorch kernel registration conflicts when diffusers
attempts to re-import the module.
On ROCm systems without proper binaries, bitsandbytes registers kernels during
import then fails. When diffusers later imports it, the duplicate registration
causes: 'RuntimeError: already a kernel registered...int8_mm_dequant'
The shim pre-tests bitsandbytes import and stubs it only if broken, allowing
working installations to function normally for other nodes.
- CompatibleDiT now converts ALL model params to compute_dtype on MPS
(previously only FP8 - GGUF models had mixed FP16/BF16 causing hangs)
- Replace MPS autocast with explicit dtype conversion in VAE encode/decode
- Skip DiT autocast on MPS (CompatibleDiT handles dtype internally)
- Guard call_rope_with_stability CUDA autocast for non-CUDA devices
- Add weights_only=True to torch.load (FutureWarning fix)
- Rename FP8CompatibleDiT → CompatibleDiT
Addresses M4 Pro macOS hang at EulerSampler 0% with GGUF models
- Add validate_blockswap_config() in blockswap.py as single validation point
- Auto-disable BlockSwap on macOS (unified memory makes it meaningless)
- Improve error messages for missing dit_offload_device
- Update CLI and ComfyUI tooltips for BlockSwap and model caching
- Update README: BlockSwap macOS note, caching descriptions, attention backends
- Remove duplicate validation from dit_model_loader.py and inference_cli.py
Partially fixes#401 (M4 Pro macOS BlockSwap offload device error)
Add compatibility shims for corrupted/missing flash_attn and xformers DLLs.
Stubs include proper __spec__ to prevent importlib.util.find_spec() crashes.
Force-verify flash_attn_2_cuda at startup; fall back to SDPA if unavailable.
Adrien Toupet