hao-ai-lab-FastVideo

FastHunyuan and FastMochi: consistency distilled video diffusion models for 8x inference speedup.
First open distillation recipes for video DiT, based on PCM.
Support distilling/finetuning/inferencing state-of-the-art open video DiTs: 1. Mochi 2. Hunyuan.
Scalable training with FSDP, sequence parallelism, and selective activation checkpointing, with near linear scaling to 64 GPUs.
Memory efficient finetuning with LoRA, precomputed latent, and precomputed text embeddings.

Dev in progress and highly experimental.

🎥 More Demos

Fast-Hunyuan comparison with original Hunyuan, achieving an 8X diffusion speed boost with the FastVideo framework.

https://github.com/user-attachments/assets/064ac1d2-11ed-4a0c-955b-4d412a96ef30

Comparison between OpenAI Sora, original Hunyuan and FastHunyuan

https://github.com/user-attachments/assets/d323b712-3f68-42b2-952b-94f6a49c4836

Change Log

2024/12/17: FastVideo v0.1 is released.

🔧 Installation

The code is tested on Python 3.10.0, CUDA 12.1 and H100.

./env_setup.sh fastvideo

🚀 Inference

We recommend using a GPU with 80GB of memory. To run the inference, use the following command:

FastMochi

# Download the model weight
python scripts/huggingface/download_hf.py --repo_id=FastVideo/FastMochi-diffusers --local_dir=data/FastMochi-diffusers --repo_type=model
# CLI inference
bash scripts/inference/inference_mochi_sp.sh

FastHunyuan

# Download the model weight
python scripts/huggingface/download_hf.py --repo_id=FastVideo/FastHunyuan --local_dir=data/FastHunyuan --repo_type=model
# CLI inference
sh scripts/inference/inference_hunyuan.sh

You can also inference FastHunyuan in the official Hunyuan github.

Distillation

Please refer to the distillation guide.

Finetuning

Please refer to the finetuning guide.

Acknowledgement

We learned and reused code from the following projects: PCM, diffusers, OpenSoraPlan, and xDiT.

We thank MBZUAI and Anyscale for their support throughout this project.