Ishan VaishandClaude Sonnet 5
7f7431779e
Wired up FastVideo/LingBot-World-Fast-Diffusers, the old LingBot-World repo's distilled causal I2V checkpoint that was never ported. Reused the existing LingBot World 2 causal-fast DiT unchanged (verified all 1421 parameter names and shapes against the released checkpoint) and added the arch/pipeline configs, registry entries, preset, pipeline subclass, example, and SSIM test needed to run it. Fixed a model-family detector collision where the LingBot World (Cam) detector would have wrongly claimed this checkpoint's path and model_index name, since it only excluded the "causal-fast" spelling. Fixed WanCrossAttention in the shared LingBot World 2 causal-fast DiT to route through the attention() dispatcher instead of calling flash_attention() directly, so the already-advertised TORCH_SDPA backend actually works when flash-attn isn't installed. Self-attention already used this dispatcher; cross-attention did not. Verified end to end: weights load (18.54B params, 16 shards) and generation produces a coherent video with correct prompt-following camera motion. Not yet verified against the original implementation's numerics (activation-trace parity) or via the SSIM regression test, both of which need hardware unavailable in this environment. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
41 lines
1.3 KiB
Python
41 lines
1.3 KiB
Python
from fastvideo import VideoGenerator
|
|
|
|
OUTPUT_PATH = "video_samples_lingbotworld_fast"
|
|
|
|
|
|
def main():
|
|
generator = VideoGenerator.from_pretrained(
|
|
"FastVideo/LingBot-World-Fast-Diffusers",
|
|
num_gpus=1,
|
|
use_fsdp_inference=False, # set to True if GPU is out of memory
|
|
dit_cpu_offload=True,
|
|
vae_cpu_offload=False,
|
|
text_encoder_cpu_offload=True,
|
|
pin_cpu_memory=True,
|
|
)
|
|
|
|
prompt = (
|
|
"The video presents a soaring journey through a fantasy jungle. The "
|
|
"wind whips past the rider's blue hands gripping the reins, causing "
|
|
"the leather straps to vibrate. The ancient gothic castle approaches "
|
|
"steadily, its stone details becoming clearer against the backdrop of "
|
|
"floating islands and distant waterfalls.")
|
|
image_path = ("https://raw.githubusercontent.com/Robbyant/lingbot-world/"
|
|
"main/examples/00/image.jpg")
|
|
action_path = "examples/inference/basic/lingbotworld_examples/00"
|
|
|
|
generator.generate_video(
|
|
prompt,
|
|
image_path=image_path,
|
|
action_path=action_path,
|
|
output_path=OUTPUT_PATH,
|
|
save_video=True,
|
|
num_frames=81,
|
|
height=480,
|
|
width=832,
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|