hao-ai-lab-FastVideo/apps/dreamverse

Dreamverse

Dreamverse is the FastVideo realtime video generation & editing platform. It lives in this monorepo under apps/dreamverse/.

Deploy on: local GPU · self-hosted B200 (SSH) · Docker · Modal

Install Dreamverse

You can install Dreamverse using one of the methods below.

Method 1: With uv pip

pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
UV_TORCH_BACKEND=cu126 uv pip install "fastvideo[dreamverse]"

Method 2: From source

git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
UV_TORCH_BACKEND=cu126 uv pip install -e ".[dreamverse]"

Use UV_TORCH_BACKEND=cu130 instead on CUDA 13.

Method 3: Using Docker

git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

apps/dreamverse/docker/docker_build.sh

See apps/dreamverse/docker/README.md for Docker build and run option details.

Optional: Building FFmpeg For Better Performance

For full streaming performance in a non-Docker install, build a custom FFmpeg binary from a FastVideo source checkout. The command below is repo-relative, so run it from the repository root:

bash apps/dreamverse/scripts/install_native_ffmpeg.sh

The installer supports Linux x86_64 and aarch64. It prefers conda-forge triplet compilers when those commands are on PATH, otherwise it falls back to system gcc/g++ (plain venv). On x86_64, x264's hand-tuned SIMD also requires nasm; install via whichever path fits your host:

sudo apt install nasm                       # Debian/Ubuntu
conda install -c conda-forge nasm           # inside an active conda env

No sudo and no conda? Build nasm from source (~30s, installs into $HOME):

(
  mkdir -p "$HOME/src" "$HOME/opt" && cd "$HOME/src"
  curl -fsSL -O https://www.nasm.us/pub/nasm/releasebuilds/2.16.03/nasm-2.16.03.tar.gz
  tar -xf nasm-2.16.03.tar.gz && cd nasm-2.16.03
  ./configure --prefix="$HOME/opt/nasm" && make -j"$(nproc)" && make install
)
export PATH="$HOME/opt/nasm/bin:$PATH"      # add to ~/.bashrc to persist

The installer writes to ~/opt/ffmpeg-native/ and emits apps/dreamverse/scripts/ffmpeg-env.sh. Source it before starting the backend so Dreamverse uses the custom FFmpeg binary:

source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server

Docker images already run this FFmpeg build during image creation and source the generated environment file at container startup.

Launch Dreamverse

Start the backend with the installed Dreamverse commands:

dreamverse-server --port 8009
dreamverse-mock-server --port 8009

Run Dreamverse with FastH3

Select the VSA data-free FastH3 Preview profile when you start the backend:

DREAMVERSE_MODEL_ID=fast-h3 dreamverse-server --port 8009

The fast-h3 profile uses four visible GPUs by default. It loads the MiniMaxAI/MiniMax-H3 base checkpoint and the vsa-datafree/adapter_model.safetensors adapter from FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA. Each request generates a 124-frame, 768×1344 video with synchronized audio and five sigma-grid points. Dreamverse uses the last frame of each segment as first-frame conditioning for the following segment.

Set CUDA_VISIBLE_DEVICES when you need to choose the four physical GPUs:

CUDA_VISIBLE_DEVICES=0,1,2,3 DREAMVERSE_MODEL_ID=fast-h3 dreamverse-server --port 8009

Expect a slow first boot. With torch.compile and startup warmup enabled (the default), the backend compiles the segment 1 and segment 2 inference paths before it reports ready — this can take tens of minutes on a cold cache, regardless of how you deploy (local, server, Docker, or Modal). /healthz responds as soon as the process is up; /readyz stays 503 until warmup finishes. To defer compilation until the first generated request while testing, set FASTVIDEO_ENABLE_STARTUP_WARMUP=0 before starting the backend.

Frontend Setup

Install the web dependencies once from the FastVideo checkout:

cd apps/dreamverse/web
npm ci

The frontend package uses package-lock.json; use npm for installs and scripts.

Quick Start: Local GPU

Start Backend

Export the API keys used for prompt rewrite and prompt enhancement:

export CEREBRAS_API_KEY=...
export GROQ_API_KEY=...

If you built the optional native FFmpeg binary above, source its environment file in the same shell before starting the backend:

source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server --host 0.0.0.0 --port 8009

The Dreamverse backend defaults to 0.0.0.0:8009 and starts one GPU worker on the first visible GPU by default.

Cosmos Predict2.5 DFD continuation (experimental)

Dreamverse can combine two converted Cosmos Predict2.5 2B packages: the distilled Text2World student creates an unconditioned first segment, then the Data-Forcing Distillation (DFD) Video2World student conditions each later segment on the prior terminal frame. Point the runtime at both local converted packages:

export DREAMVERSE_MODEL_ID=cosmos25-dfd
export DREAMVERSE_MODEL_PATH=/path/to/Cosmos-Predict2.5-2B-Distilled-TrigFlow-FastVideo
export DREAMVERSE_COSMOS25_DFD_MODEL_PATH=/path/to/Cosmos-Predict2.5-2B-DFD-FastVideo
export ENABLE_TORCH_COMPILE=0
dreamverse-server --host 0.0.0.0 --port 8009

The backend loads and warms both model roles before reporting ready. Both use BF16, Torch SDPA, 704x1280 output, 24 FPS, and four steps. Bootstrap segments contain 77 frames. DFD segments contain 81 decoded frames, but Dreamverse drops the repeated conditioning frame before streaming, leaving 80 new frames. An initial user image selects DFD immediately without treating that first frame as a cross-segment overlap.

The profile uses a 30-minute session lease because sequential generation on GB10-class hardware can exceed Dreamverse's five-minute default while the GPU is still making progress. Deployments can override the lease with FASTVIDEO_SESSION_TIMEOUT_SECONDS.

Cosmos does not produce audio, so the backend supplies duration-matched silent 24 kHz audio for the existing browser streaming contract and trims 1,000 audio samples with each repeated DFD boundary frame. Runtime LoRA changes are not supported. Full segments take roughly 145 seconds on GB10, so this profile is a continuation-quality integration rather than a real-time configuration.

Check Readiness

In another shell, verify that the backend process is alive:

curl http://localhost:8009/healthz

Then wait for GPU workers and startup warmup to finish:

curl http://localhost:8009/readyz

You can also run the same readiness path with:

BACKEND_HOST=localhost BACKEND_PORT=8009 apps/dreamverse/scripts/smoke_local.sh

If a backend is already running and you only want the script to probe it:

DREAMVERSE_SMOKE_START_BACKEND=0 apps/dreamverse/scripts/smoke_local.sh

Start Frontend

Start the frontend:

cd apps/dreamverse/web
BACKEND_HOST=localhost BACKEND_PORT=8009 npm run dev

Open http://localhost:5299.

Server B200 deployment (SSH)

Deploying on a remote GPU host (for example a B200 box) is a local install run over SSH, plus a few server-specific concerns. Two paths:

Option A: Native (source install)

SSH in, then follow Install → From source and (recommended) Building FFmpeg, then start the backend as in Quick Start: Local GPU. For a remote host, a few things differ from localhost:

• Bind all interfaces: dreamverse-server --host 0.0.0.0 --port 8009.
• Point the frontend/client at the host: BACKEND_HOST=<b200-host> BACKEND_PORT=8009 npm run dev.
• Keep the backend alive across SSH sessions (tmux / systemd / nohup).
• Expose / firewall port 8009, or front it with a reverse proxy + auth.

Option B: Docker (on the server)

SSH in, then follow Install → Using Docker and the run steps in docker/README.md:

CEREBRAS_API_KEY="<key>" GROQ_API_KEY="<key>" apps/dreamverse/docker/docker_run.sh

Quick Start: Mock Backend (For UI development)

The mock server emulates the Dreamverse backend protocol and streams a synthetic FFmpeg-generated fMP4 clip, so the frontend can run without a GPU.

dreamverse-mock-server --latency 200 --port 8009

Tests

Run the focused backend tests that validate local startup wiring, config, GPU selection, and mock-server behavior:

pytest apps/dreamverse/dreamverse/tests/test_config.py \
  apps/dreamverse/dreamverse/tests/test_entrypoints.py \
  apps/dreamverse/dreamverse/tests/test_gpu_pool.py \
  apps/dreamverse/dreamverse/tests/test_minimax_h3_generation.py \
  apps/dreamverse/dreamverse/tests/test_mock_server.py -q

Run the broader Dreamverse backend suite:

pytest apps/dreamverse/dreamverse/tests -q

Run the frontend tests:

cd apps/dreamverse/web
npm test

Run the frontend e2e tests:

cd apps/dreamverse/web
npm run e2e

Troubleshooting

dreamverse-server exits with an install hint

• install the Dreamverse extra with UV_TORCH_BACKEND=cu126 uv pip install -e ".[dreamverse]" from a source checkout, or UV_TORCH_BACKEND=cu126 uv pip install "fastvideo[dreamverse]" from PyPI (cu130 on CUDA 13).

Prompt-provider environment variable errors

• set CEREBRAS_API_KEY
• set GROQ_API_KEY
• direct dreamverse-server launches do not source ~/.env; export the keys in the shell or use the bundled launch scripts, which source ~/.env.

/readyz stays at 503

• wait for model loading and startup warmup to finish
• confirm a compatible CUDA GPU is visible to the process
• check backend logs for worker startup or warmup failures
• for a startup/debug pass without warmup, set FASTVIDEO_ENABLE_STARTUP_WARMUP=0 before starting the backend

Only one GPU is used

• this is the default local behavior
• set FASTVIDEO_GPU_COUNT=<N> to start N GPU worker subprocesses inside one backend instance
• set FASTVIDEO_GPU_COUNT=all to start one worker for every visible GPU
• use CUDA_VISIBLE_DEVICES first if you need to pin the visible GPU set

Frontend cannot connect to backend

• confirm the backend is running on 8009; if not, point the frontend at it with BACKEND_HOST=<host> BACKEND_PORT=<port> npm run dev
• confirm http://localhost:8009/healthz responds before starting the frontend
• confirm http://localhost:8009/readyz returns 200 before clicking Generate
• use apps/dreamverse/scripts/smoke_local.sh for a repeatable local startup check

Mock backend fails during startup

• install FFmpeg or set FASTVIDEO_FFMPEG_BIN to an FFmpeg binary
• for non-mock local GPU streaming performance, use the native FFmpeg installer described above

Notes

Dreamverse owns its backend app under apps/dreamverse/dreamverse/. It expects dreamverse-server, not fastvideo serve.