fix: ensure model.device is set in ModelPatcher.partially_load

Assign self.model.device = device_to during DisTorch V2 partially_load so the model's device reflects the target allocation after loading.

AssertionError: Input tensors must be on cuda.
Fixes #119

Possible issue when used with custom samplers
Fixes #130
This commit is contained in:
John Pollock
2025-10-15 21:37:33 -05:00
parent 35e81e92a8
commit c713f637b0
+2
View File
@@ -301,6 +301,8 @@ def register_patched_safetensor_modelpatcher():
self.model.current_weight_patches_uuid = self.patches_uuid
self.model.device = device_to
logger.info("[MultiGPU DisTorch V2] DisTorch loading completed.")
logger.info(f"[MultiGPU DisTorch V2] Total memory: {mem_counter / (1024 * 1024):.2f}MB")