# Dreamverse

Dreamverse is the FastVideo realtime video generation & editing platform. It lives in this monorepo under `apps/dreamverse/`.

**Deploy on:** [local GPU](#quick-start-local-gpu) · [self-hosted B200 (SSH)](#server-b200-deployment-ssh) · [Docker](docker/README.md) · [Modal](scripts/modal/README.md)

## Install Dreamverse

You can install Dreamverse using one of the methods below.

### Method 1: With uv pip

```bash
pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
UV_TORCH_BACKEND=cu126 uv pip install "fastvideo[dreamverse]"
```

### Method 2: From source

```bash
git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
UV_TORCH_BACKEND=cu126 uv pip install -e ".[dreamverse]"
```

Use `UV_TORCH_BACKEND=cu130` instead on CUDA 13.

### Method 3: Using Docker

```bash
git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

apps/dreamverse/docker/docker_build.sh
```

See `apps/dreamverse/docker/README.md` for Docker build and run option details.

## Optional: Building FFmpeg For Better Performance

For full streaming performance in a non-Docker install, build a custom FFmpeg
binary from a FastVideo source checkout. The command below is repo-relative,
so run it from the repository root:

```bash
bash apps/dreamverse/scripts/install_native_ffmpeg.sh
```

The installer supports Linux `x86_64` and `aarch64`. It prefers conda-forge
triplet compilers when those commands are on `PATH`, otherwise it falls back to
system `gcc`/`g++` (plain venv). On `x86_64`, x264's hand-tuned SIMD also
requires `nasm`; install via whichever path fits your host:

```bash
sudo apt install nasm                       # Debian/Ubuntu
conda install -c conda-forge nasm           # inside an active conda env
```

No sudo and no conda? Build `nasm` from source (~30s, installs into `$HOME`):

```bash
(
  mkdir -p "$HOME/src" "$HOME/opt" && cd "$HOME/src"
  curl -fsSL -O https://www.nasm.us/pub/nasm/releasebuilds/2.16.03/nasm-2.16.03.tar.gz
  tar -xf nasm-2.16.03.tar.gz && cd nasm-2.16.03
  ./configure --prefix="$HOME/opt/nasm" && make -j"$(nproc)" && make install
)
export PATH="$HOME/opt/nasm/bin:$PATH"      # add to ~/.bashrc to persist
```

The installer writes to `~/opt/ffmpeg-native/` and emits
`apps/dreamverse/scripts/ffmpeg-env.sh`. Source it before starting the backend
so Dreamverse uses the custom FFmpeg binary:

```bash
source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server
```

Docker images already run this FFmpeg build during image creation and source the
generated environment file at container startup.

## Launch Dreamverse

Start the backend with the installed Dreamverse commands:

```bash
dreamverse-server --port 8009
dreamverse-mock-server --port 8009
```

> **Expect a slow first boot.** With `torch.compile` and startup warmup enabled
> (the default), the backend compiles the segment 1 and segment 2 inference
> paths before it reports ready — this can take **tens of minutes on a cold
> cache**, regardless of how you deploy (local, server, Docker, or Modal).
> `/healthz` responds as soon as the process is up; `/readyz` stays `503` until
> warmup finishes. For a faster, uncompiled startup while testing, set
> `FASTVIDEO_ENABLE_STARTUP_WARMUP=0` before starting the backend.

## Frontend Setup

Install the web dependencies once from the FastVideo checkout:

```bash
cd apps/dreamverse/web
npm ci
```

The frontend package uses `package-lock.json`; use npm for installs and scripts.

## Quick Start: Local GPU

### Start Backend

Export the API keys used for prompt rewrite and prompt enhancement:

```bash
export CEREBRAS_API_KEY=...
export GROQ_API_KEY=...
```

If you built the optional native FFmpeg binary above, source its environment
file in the same shell before starting the backend:

```bash
source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server --host 0.0.0.0 --port 8009
```

The Dreamverse backend defaults to `0.0.0.0:8009` and starts one GPU worker on
the first visible GPU by default.

### Check Readiness

In another shell, verify that the backend process is alive:

```bash
curl http://localhost:8009/healthz
```

Then wait for GPU workers and startup warmup to finish:

```bash
curl http://localhost:8009/readyz
```

You can also run the same readiness path with:

```bash
BACKEND_HOST=localhost BACKEND_PORT=8009 apps/dreamverse/scripts/smoke_local.sh
```

If a backend is already running and you only want the script to probe it:

```bash
DREAMVERSE_SMOKE_START_BACKEND=0 apps/dreamverse/scripts/smoke_local.sh
```

### Start Frontend

Start the frontend:

```bash
cd apps/dreamverse/web
BACKEND_HOST=localhost BACKEND_PORT=8009 npm run dev
```

Open `http://localhost:5299`.

## Server B200 deployment (SSH)

Deploying on a remote GPU host (for example a B200 box) is a local install run
over SSH, plus a few server-specific concerns. Two paths:

### Option A: Native (source install)

SSH in, then follow [Install → From source](#method-2-from-source) and
(recommended) [Building FFmpeg](#optional-building-ffmpeg-for-better-performance),
then start the backend as in [Quick Start: Local GPU](#quick-start-local-gpu).
For a remote host, a few things differ from localhost:

- Bind all interfaces: `dreamverse-server --host 0.0.0.0 --port 8009`.
- Point the frontend/client at the host: `BACKEND_HOST=<b200-host> BACKEND_PORT=8009 npm run dev`.
- Keep the backend alive across SSH sessions (`tmux` / `systemd` / `nohup`).
- Expose / firewall port `8009`, or front it with a reverse proxy + auth.

### Option B: Docker (on the server)

SSH in, then follow [Install → Using Docker](#method-3-using-docker) and the run
steps in [`docker/README.md`](docker/README.md):

```bash
CEREBRAS_API_KEY="<key>" GROQ_API_KEY="<key>" apps/dreamverse/docker/docker_run.sh
```

## Quick Start: Mock Backend (For UI development)

The mock server emulates the Dreamverse backend protocol and streams a
synthetic FFmpeg-generated fMP4 clip, so the frontend can run without a GPU.

```bash
dreamverse-mock-server --latency 200 --port 8009
```

## Tests

Run the focused backend tests that validate local startup wiring, config, GPU
selection, and mock-server behavior:

```bash
pytest apps/dreamverse/dreamverse/tests/test_config.py \
  apps/dreamverse/dreamverse/tests/test_entrypoints.py \
  apps/dreamverse/dreamverse/tests/test_gpu_pool.py \
  apps/dreamverse/dreamverse/tests/test_mock_server.py -q
```

Run the broader Dreamverse backend suite:

```bash
pytest apps/dreamverse/dreamverse/tests -q
```

Run the frontend tests:

```bash
cd apps/dreamverse/web
npm test
```

Run the frontend e2e tests:

```bash
cd apps/dreamverse/web
npm run e2e
```

## Troubleshooting

`dreamverse-server` exits with an install hint

- install the Dreamverse extra with `UV_TORCH_BACKEND=cu126 uv pip install -e ".[dreamverse]"` from a
  source checkout, or `UV_TORCH_BACKEND=cu126 uv pip install "fastvideo[dreamverse]"` from PyPI
  (`cu130` on CUDA 13).

Prompt-provider environment variable errors

- set `CEREBRAS_API_KEY`
- set `GROQ_API_KEY`
- direct `dreamverse-server` launches do not source `~/.env`; export the keys
  in the shell or use the bundled launch scripts, which source `~/.env`.

`/readyz` stays at `503`

- wait for model loading and startup warmup to finish
- confirm a compatible CUDA GPU is visible to the process
- check backend logs for worker startup or warmup failures
- for a startup/debug pass without warmup, set
  `FASTVIDEO_ENABLE_STARTUP_WARMUP=0` before starting the backend

Only one GPU is used

- this is the default local behavior
- set `FASTVIDEO_GPU_COUNT=<N>` to start N GPU worker subprocesses inside one
  backend instance
- set `FASTVIDEO_GPU_COUNT=all` to start one worker for every visible GPU
- use `CUDA_VISIBLE_DEVICES` first if you need to pin the visible GPU set

Frontend cannot connect to backend

- confirm the backend is running on `8009`; if not, point the frontend at it
  with `BACKEND_HOST=<host> BACKEND_PORT=<port> npm run dev`
- confirm `http://localhost:8009/healthz` responds before starting the frontend
- confirm `http://localhost:8009/readyz` returns `200` before clicking Generate
- use `apps/dreamverse/scripts/smoke_local.sh` for a repeatable local startup
  check

Mock backend fails during startup

- install FFmpeg or set `FASTVIDEO_FFMPEG_BIN` to an FFmpeg binary
- for non-mock local GPU streaming performance, use the native FFmpeg installer
  described above

## Notes

Dreamverse owns its backend app under `apps/dreamverse/dreamverse/`. It expects
`dreamverse-server`, not `fastvideo serve`.