[perf]: Load H3 DiT weights onto CUDA, not CPU
GB10 unified memory made the CPU-then-copy path take minutes for the 35B FastH3 transformer. When host offload is off, read safetensors onto the accelerator.
This commit is contained in:
committed by
Satyam Srivastava
parent
a3d9725d15
commit
cc9bfd069d
@@ -161,6 +161,12 @@ is power-cycled. To avoid it:
|
||||
on: "CPU" offload uses the same unified RAM. Multi-GPU FSDP sharding remains
|
||||
available because it partitions weights without parking them in a separate
|
||||
host pool.
|
||||
- **MiniMax H3 / FastH3** still needs sequential loading on one GB10. The Qwen3-VL
|
||||
conditioner is tens of gigabytes of BF16. If the DiT and VAEs load while that
|
||||
encoder is still resident, the process is a typical `earlyoom` kill (Python is
|
||||
preferred). The CUDA pipeline now encodes first, releases the encoder, then
|
||||
loads DiT and VAEs onto the accelerator (`to_cpu` follows `cpu_offload`, which
|
||||
is off here). See [Offloading](../../inference/offloading.md).
|
||||
|
||||
## Gotchas specific to the GB10
|
||||
|
||||
|
||||
Reference in New Issue
Block a user