907620ff22 [docs] dreamverse-integration: drift audit + integration tradeoff doc
Comprehensive review against ../Dreamverse and ../FastVideo-internal
plus 4-option integration tradeoff analysis with recommendation.

Driven by user request to (1) verify will/ltx2_sr_port has not drifted
from the integration goal, and (2) analyze whether Dreamverse should
remain a separate repo, become a subfolder, or merge into FastVideo's
namespace entirely.

Methodology: 4 parallel exploration agents mapped Dreamverse repo
structure, FastVideo-internal residual, FastVideo public cross-repo
surfaces, and OSS service-in-monorepo precedents (vLLM, BentoML, Ray
Serve, TGI+ChatUI, Transformers.js, ComfyUI, AUTOMATIC1111). Synthesis
authored by ultrabrain category. Oracle review applied as
PASS-WITH-FIXES; critical issues (broad zero-drift wording, stale Dynamo
mapping citation, broken Ray Serve URL, missing D-8/VPO/D-12-A/D-12-B/
SBS coverage) addressed in this commit.

* integration-review.md (new, 863 lines): Part 1 drift audit, Part 2
  integration tradeoffs, Part 3 action items.

  Part 1 — Drift audit
    - Methodology: typed-public-boundary criterion, intentional refactor
      vs drift, evidence sourced from worktree files + memory dir.
    - Zero core typed API drift on the integration surface — but the
      realtime-runtime contract surface (health routes) IS real drift.
    - 8 numbered drift items (Dreamverse README/bootstrap, prompt
      enhancer fork, cerebras_ifm gap, health routes, layerwise offload,
      upsampler CLI, missing example config, LTX-2 stage equivalence
      verification gap).
    - 6 deferred/accepted residual items.
    - 17-row drift summary table with priority / effort / status /
      tracked-where / next-action columns. Includes D-8 (ltx2_image_crf
      flow), VPO (video_position_offset_sec semantics — overdue),
      D-12-A (GpuPool docstring), D-12-B (run_async migration), SBS
      (session/blob lifecycle).

  Part 2 — Integration path tradeoffs
    - Option A: status quo (Dreamverse separate, depends on fastvideo).
    - Option B: Dreamverse as subfolder under FastVideo.
    - Option C: full merge into fastvideo.entrypoints.dreamverse.*.
    - Option D: hybrid — backend merges, frontend stays separate.
    - Comparison matrix on 13 axes.
    - 7 OSS precedent rows with citations (vLLM, BentoML, Ray Serve,
      TGI, Transformers.js, ComfyUI, AUTOMATIC1111).
    - Recommendation: Option D (constrained) — backend merges as
      generic FastVideo streaming, frontend stays separate. Mirrors
      ComfyUI's late-stage frontend split. Strong precedent: TGI +
      ChatUI. Critical librarian finding: NO 1:1 precedent exists for
      "Python ML library + Next.js product merged into library
      namespace" — argues against Option C.
    - Conditions that would change the recommendation enumerated.

  Part 3 — Action items: phased migration sketch (phases 0-5) with
  concrete file paths, effort estimates, and cross-references to
  open-threads.md / decisions-log.md.

* README.md: register integration-review.md in deep-dive reading guide.
  Bump Last reconciled to b36bdbc9 (current PR #1288 head, post
  STACK.md removal + integration-review.md addition); 36 commits ahead.

* state.md: branch tip refresh — b36bdbc9 / 36 commits / now includes
  integration-review.md.

* pr-roadmap.md: PR #1288 head bumped to b36bdbc9.

* open-threads.md: Last updated header bumped to b36bdbc9. Item D
  reference SHA bumped to b36bdbc9.

* authors.md: PR #1288 row bumped to b36bdbc9 / 36 commits / 70 files
  (post STACK.md removal).

Verified:
  - Pre-commit clean (yapf/ruff/mypy/codespell skipped per memory dir
    excludes; spaces-check passes).
  - Oracle review: PASS-WITH-FIXES, all 4 critical issues addressed.
  - Cross-references resolve: design.md, cross-repo-surfaces.md,
    decisions-log.md, open-threads.md, pr-roadmap.md, state.md,
    runbook.md, authors.md, streaming-server.md, quantization.md.

PR #1288: https://github.com/hao-ai-lab/FastVideo/pull/1288

Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-05 04:20:56 -07:00
2026-02-23 15:14:42 -08:00
2026-04-02 16:07:24 -07:00
2026-04-02 16:07:24 -07:00
2026-04-02 16:07:24 -07:00
2026-04-02 16:07:24 -07:00
2026-02-10 01:11:17 -08:00
2025-02-20 11:33:58 -08:00
2026-03-17 14:26:07 -07:00
2025-11-27 00:37:03 -08:00

| Documentation | Quick Start | Weekly Dev Meeting | 🟣💬 Slack | 🟣💬 WeChat |

FastVideo is a unified post-training and real-time inference framework for accelerated video generation.

NEWS

More News

Key Features

FastVideo has the following features:

  • End-to-end post-training support for bidirectional and autoregressive models:
    • Support full finetuning and LoRA finetuning for state-of-the-art open video DiTs
    • Data preprocessing pipeline for video, image, and text data
    • Distribution Matching Distillation (DMD2) stepwise distillation.
    • Sparse attention with Video Sparse Attention
    • Sparse distillation to achieve >50x denoising speedup
    • Scalable training with FSDP2, sequence parallelism, and selective activation checkpointing.
    • Causal distillation through Self-Forcing
    • See this page for full list of supported models and recipes.
  • State-of-the-art performance optimizations for inference
    • Sequence Parallelism for distributed inference
    • Multiple state-of-the-art attention backends
    • User-friendly CLI and Python API
    • See this page for full list of supported optimizations.
  • Diverse hardware and OS support
    • Support H100, A100, 4090
    • Support Linux, Windows, MacOS
    • See this page for full list of supported models, hardware assumptions, and optimization compatibility.

Getting Started

We recommend using uv to create a clean environment. If you previously used Conda, switching to uv generally gives faster and more stable installs.

# Create and activate a new uv environment
uv venv --python 3.12 --seed
source .venv/bin/activate

# Install FastVideo
uv pip install fastvideo

Please see our docs for more detailed installation instructions.

Sparse Distillation

For our sparse distillation techniques, please see our distillation docs and check out our blog.

See below for recipes and datasets:

Model Sparse Distillation Dataset
FastWan2.1-T2V-1.3B Recipe FastVideo Synthetic Wan2.1 480P
FastWan2.2-TI2V-5B Recipe FastVideo Synthetic Wan2.2 720P

Inference

Generating Your First Video

Here's a minimal example to generate a video using the default settings. Make sure VSA kernels are installed. Create a file called example.py with the following code:

import os
from fastvideo import VideoGenerator

def main():
    os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "VIDEO_SPARSE_ATTN"

    # Create a video generator with a pre-trained model
    generator = VideoGenerator.from_pretrained(
        "FastVideo/FastWan2.1-T2V-1.3B-Diffusers",
        num_gpus=1,  # Adjust based on your hardware
    )

    # Define a prompt for your video
    prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."

    # Generate the video
    video = generator.generate_video(
        prompt,
        output_path="my_videos/",  # Controls where videos are saved
        save_video=True
    )

if __name__ == '__main__':
    main()

Run the script with:

python example.py

For a more detailed guide, please see our inference quick start.

Awesome work using FastVideo or our research projects

  • SGLang: SGLang's diffusion inference functionality is based on a fork of FastVideo on Sept. 24, 2025.
  • DanceGRPO: A unified framework to adapt Group Relative Policy Optimization (GRPO) to visual generation paradigms. Code based on FastVideo.
  • SRPO: A method to directly align the full diffusion trajectory with fine-grained human preference. Code based on FastVideo.
  • DCM: Dual-expert consistency model for efficient and high-quality video generation. Code based on FastVideo.
  • HY-WorldPlay: An action-conditioned world model model trained using FastVideo framework.
  • Hunyuan Video 1.5: A leading lightweight video generation model, where they proposed SSTA based on Sliding Tile Attention.
  • Kandinsky-5.0: A family of diffusion models for video & image generation, where their NABLA attention includes a Sliding Tile Attention branch.
  • LongCat Video: A foundational video generation model with 13.6B parameters with block-sparse attention similar to Video Sparse Attention.

🤝 Contributing

We welcome all contributions. Please check out our guide here. See details in development roadmap.

Acknowledgement

We learned the design and reused code from the following projects: Wan-Video, ThunderKittens, DMD2, diffusers, xDiT, vLLM, SGLang. We thank MBZUAI, Anyscale, and GMI Cloud for their support throughout this project.

Citation

If you find FastVideo useful, please consider citing our research work:

@article{zhang2025vsa,
  title={Vsa: Faster video diffusion with trainable sparse attention},
  author={Zhang, Peiyuan and Chen, Yongqi and Huang, Haofeng and Lin, Will and Liu, Zhengzhong and Stoica, Ion and Xing, Eric and Zhang, Hao},
  journal={arXiv preprint arXiv:2505.13389},
  year={2025}
}

@article{zhang2025fast,
  title={Fast video generation with sliding tile attention},
  author={Zhang, Peiyuan and Chen, Yongqi and Su, Runlong and Ding, Hangliang and Stoica, Ion and Liu, Zhengzhong and Zhang, Hao},
  journal={arXiv preprint arXiv:2502.04507},
  year={2025}
}
S
Description
No description provided
Readme
438 MiB
Languages
Python 89.6%
TypeScript 5.6%
Cuda 2%
C++ 1.1%
Shell 0.7%
Other 0.9%