[bugfix]: don't let a stale HF-passthrough marker override a moved encoder (#1339)

_fastvideo_input_device is stamped at load time with the loader's target
device, which is "cpu" when text_encoder_cpu_offload is on. encode_text then
moves such an encoder to the compute device for the forward, after which the
marker is stale. Honouring it fed cpu token ids to cuda weights:

  RuntimeError: Expected all tensors to be on the same device, but got index
  is on cpu, different from other tensors on cuda:0

Seen on FLUX.2 Klein, whose Qwen3 text encoder takes the HF-passthrough path.
Once the encoder has been moved, its param device is authoritative; the marker
only speaks when nothing moved and the module has no parameters of its own.
This commit is contained in:
Raghav
2026-07-13 17:26:46 -07:00
committed by SolitaryThinker
parent 35e55aa7f6
commit 70bd4bb756
+13 -6
View File
@@ -221,12 +221,19 @@ class TextEncodingStage(PipelineStage):
encoder_device = torch.device(target_device)
moved_for_forward = True
# An explicit `device=` wins. Otherwise HF-passthrough encoders pin
# their own placement via _fastvideo_input_device (they return before
# the FSDP wrapping path, so encoder_device cannot speak for them);
# every other encoder follows its (possibly just-moved) param device.
input_device = (torch.device(target_device) if device is not None else getattr(
text_encoder, "_fastvideo_input_device", encoder_device))
# An explicit `device=` wins. Otherwise follow the encoder's real
# param device. Once it has been moved for the forward that is the
# target device, and an HF-passthrough encoder's
# _fastvideo_input_device (stamped at load, e.g. "cpu" under
# text_encoder_cpu_offload) is stale -- honouring it would feed cpu
# token ids to cuda weights. The marker only speaks when nothing
# moved and the module has no parameters to speak for it.
if device is not None:
input_device = torch.device(target_device)
elif moved_for_forward:
input_device = encoder_device
else:
input_device = getattr(text_encoder, "_fastvideo_input_device", encoder_device)
tok_kwargs = dict(encoder_config.tokenizer_kwargs)
if max_length is not None: