[perf]: Load H3 DiT weights onto CUDA, not CPU

GB10 unified memory made the CPU-then-copy path take minutes for
the 35B FastH3 transformer. When host offload is off, read
safetensors onto the accelerator.
This commit is contained in:
Aryan Kumar
2026-08-30 23:27:24 -07:00
committed by Satyam Srivastava
parent a3d9725d15
commit cc9bfd069d
3 changed files with 21 additions and 1 deletions
@@ -161,6 +161,12 @@ is power-cycled. To avoid it:
on: "CPU" offload uses the same unified RAM. Multi-GPU FSDP sharding remains
available because it partitions weights without parking them in a separate
host pool.
- **MiniMax H3 / FastH3** still needs sequential loading on one GB10. The Qwen3-VL
conditioner is tens of gigabytes of BF16. If the DiT and VAEs load while that
encoder is still resident, the process is a typical `earlyoom` kill (Python is
preferred). The CUDA pipeline now encodes first, releases the encoder, then
loads DiT and VAEs onto the accelerator (`to_cpu` follows `cpu_offload`, which
is off here). See [Offloading](../../inference/offloading.md).
## Gotchas specific to the GB10