2d6e7d9b6b
Co-authored-by: magicbear <magicbear@users.noreply.github.com> Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
90 lines
3.8 KiB
Markdown
90 lines
3.8 KiB
Markdown
# MiniMax H3 on NVIDIA DGX Spark (GB10)
|
|
|
|
This guide covers running MiniMax H3 / FastH3 inference on DGX Spark clusters
|
|
(GB10, 128 GB unified memory per node) and the **encoder split** mode that
|
|
makes 720p fit without host offload.
|
|
|
|
## Why a dedicated encoder node group
|
|
|
|
Measured resident sizes on GB10 (NVFP4 checkpoint, lazy load):
|
|
|
|
| component | size |
|
|
|---|---|
|
|
| text_encoder (Qwen3-VL) | 47.98 GiB |
|
|
| transformer (DiT) | 65.53 GiB |
|
|
| video VAE | 9.74 GiB |
|
|
| audio VAE | 0.58 GiB |
|
|
|
|
All three heavy groups resident at once is ~124 GiB, above the ~115 GiB usable
|
|
on a 128 GB GB10, so 720p (768x1344x124) OOMs on a single node no matter what
|
|
offload knobs are set. Sequential/lazy loading already avoids encoding and
|
|
denoising at the same time; the encoder split goes one step further and runs
|
|
them on **different nodes**:
|
|
|
|
- the first `N` nodes (world ranks `0..N-1`) load only the Qwen3-VL
|
|
conditioner, run `input_preparation` + `conditioning`, and NCCL-broadcast
|
|
`prompt_embeds` to everyone over the world group;
|
|
- the remaining nodes load only DiT + VAEs (never the text encoder) and form
|
|
the single sequence-parallel group, `sp_size = num_gpus - encoder_workers`.
|
|
|
|
A denoise node then peaks around ~76 GiB plus activations, which leaves plenty
|
|
of headroom for 720p.
|
|
|
|
## Enabling
|
|
|
|
The feature is off by default and only valid with the Ray backend.
|
|
|
|
```bash
|
|
export FASTVIDEO_H3_ENCODER_SPLIT=1 # or FastVideoArgs.h3_encoder_split / --h3-encoder-split
|
|
export FASTVIDEO_H3_ENCODER_NODES=1 # nodes dedicated to the encoder; default 1
|
|
```
|
|
|
|
`fastvideo generate` exposes only `--config`, so pass the same two settings as
|
|
dotted overrides into the experimental catch-all:
|
|
`--generator.pipeline.experimental.h3_encoder_split true` and
|
|
`--generator.pipeline.experimental.h3_encoder_nodes 1`.
|
|
|
|
Cluster-side prerequisites (all nodes): the FastVideo venv, the model path,
|
|
and per-node fabric exports (`NCCL_SOCKET_IFNAME` / `GLOO_SOCKET_IFNAME` /
|
|
`NCCL_IB_HCA` set before `ray start`, since the executor deliberately does not
|
|
overwrite these per-node values). Start one Ray head and join the workers as
|
|
usual; pass `--num-gpus <total nodes>` and `--execution-backend ray`.
|
|
|
|
## Choosing the encoder-node count
|
|
|
|
The denoise group runs sequence parallelism, so its size must divide the DiT
|
|
attention head count (56 in the released FastH3 checkpoints). Valid encoder
|
|
node counts are therefore constrained: with 8 nodes only `1, 4, 6, 7` are
|
|
legal (SP = 7, 4, 2, 1). This is validated before any weights load, with the
|
|
legal values listed in the error.
|
|
|
|
One request packs a single prompt presentation, so world rank 0 is the only
|
|
encoder that computes; extra encoder ranks mirror the broadcast collectives
|
|
and stay idle (reserved capacity, not a speedup for single-prompt requests).
|
|
|
|
## Verified run (8x GB10, NVFP4 checkpoint, TAEH3 preview decode)
|
|
|
|
Encoder node = rank 0, denoise SP = 7, 5 DMD steps:
|
|
|
|
| geometry | result | wall |
|
|
|---|---|---|
|
|
| 256x448x124 | ok | 29.2 s cold |
|
|
| 768x1344x124 (720p) | ok | 51.6 s cold / 36.9 s steady |
|
|
|
|
Without the split, the same 720p request reaches ~104 GiB while loading the
|
|
DiT after conditioning and OOMs inside safetensors `persistent_load`.
|
|
|
|
## Troubleshooting
|
|
|
|
- `attention heads (56) must be divisible by sequence parallel size`: the
|
|
encoder-node count left an illegal denoise-group size; use one of the values
|
|
in the message.
|
|
- Drivers hang with no stage events and node memory stuck mid-load: check the
|
|
cluster is actually free (`ray status` should show `0.0/8.0 GPU` and no
|
|
actors). A leftover job holding the placement group starves the new run, and
|
|
a full VAE decode of 720p can die at the NVRM level (`dmesg | grep NVRM`)
|
|
with no Python traceback - prefer `--video-decode-backend taeh3` at 720p on
|
|
128 GB nodes.
|
|
- The `prompt_embeds` broadcast travels over the world NCCL group, not the Ray
|
|
object store (whose 4 GB cap spills large tensors to disk).
|