Co-authored-by: sy3535 <sy3535@maxwell.poly.edu> Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
30 KiB
CI/CD Architecture
This is the canonical reference for FastVideo's CI/CD system. Contributor-facing PR steps live in Pull Requests, and test-authoring guidance lives in Testing.
Overview
FastVideo splits validation across GitHub Actions, Buildkite, Slinky Slurm, and Mergify:
PR opened or updated
|
|-- Tier 1: pre-commit
| GitHub Actions on ubuntu-latest
| style, lint, type, spelling, Markdown, workflow syntax, filenames
|
|-- Tier 2: Fastcheck
| Buildkite schedules six lanes on the Slinky Slurm cluster
| encoder, VAE, transformer, kernel, unit, DreamVerse
|
|-- /merge or ready label
|
`-- Tier 3: Change-aware merge gate
trusted base-branch planner classifies the complete PR diff
Buildkite adds only relevant integration, quality, training,
API, or performance lanes on Slinky Slurm
|
pass -> Mergify squash-merges when all merge conditions pass
fail -> fix, push, and re-run
|-- /test full
`-- Explicit all-20-lane diagnostic run
`-- weekly schedule on main
`-- Complete four-GPU SSIM matrix
CI is not one monolithic job:
- GitHub Actions owns pre-commit, slash-command handling, aggregate status updates, docs deployment, image builds, package publishing, and community automations.
- Buildkite owns the GPU test graph, statuses, and trusted dispatch control plane. Its agent runs on the Slurm login plane; it does not execute test payloads.
- Slinky Slurm is the only active CI compute backend. A host-owned dispatcher leases GPUs from a persistent four-GPU allocation and runs each lane in an isolated Enroot container at the immutable PR SHA.
- Mergify owns merge protection, labeling, and the final squash merge.
The old files under fastvideo/tests/modal/ are retained as dormant manual
rollback code. .buildkite/scripts/pr_test.sh rejects Buildkite invocations,
and no pipeline or slash-command route calls Modal.
Three Buildkite entry pipelines share the validated graph:
| Pipeline | Trigger | Scope |
|---|---|---|
pr-fastcheck |
Automatic pull-request webhook | Six Fastcheck lanes |
ci |
/merge, ready, schedules, and /test API builds |
Change-aware merge gates, scheduled SSIM, explicit Full Suite, Fastcheck reruns, or one direct lane |
fastvideo-performance-lane |
GitHub weekly scheduler | Direct performance lane |
Each entry pipeline starts with the same trusted pipeline-upload job on the
ci-runner queue. The ci pipeline's incoming GitHub webhook is disabled;
otherwise it would duplicate the automatic pr-fastcheck build. API and
scheduled builds continue to work with webhook processing disabled.
CI Tiers
Tier 1: Pre-commit
| Attribute | Value |
|---|---|
| Triggered by | Pull requests targeting main, plus /test pre-commit through workflow_call |
| Runner | GitHub Actions, ubuntu-latest |
| Workflow | .github/workflows/ci-precommit.yml |
| Local command | pre-commit run --all-files |
The workflow runs .pre-commit-config.yaml with --hook-stage manual.
| Hook | Checks |
|---|---|
yapf |
Python formatting |
ruff |
Python linting and auto-fixes |
codespell |
Spelling in code and docs |
pymarkdown |
Markdown formatting |
actionlint |
GitHub Actions workflow syntax |
mypy |
Static typing |
check-filenames |
Spaces in tracked filenames |
Run the pre-commit command above to reproduce failures locally. Do not bypass the project hook chain by calling individual tools directly unless you are debugging a hook implementation.
Tier 2: Fastcheck
| Attribute | Value |
|---|---|
| Triggered by | Buildkite PR builds with TEST_SCOPE=fastcheck or unset |
| Compute | Slinky Slurm (ci-runner queue) |
| Definition | .buildkite/pipeline.yml |
| Entrypoint | Trusted host driver -> .buildkite/scripts/unit_test.sh or .buildkite/scripts/lanes/*.sh |
Fastcheck always schedules these six lanes: encoder, VAE, transformer, custom kernels, unit tests, and DreamVerse. Static steps replace the former host-side path-filter plugin: the login plane never checks out or executes PR code.
Tier 3: Change-Aware Merge Gate
| Attribute | Value |
|---|---|
| Triggered by | /merge, adding ready, or a new push to a PR that already has ready |
| Compute | Slinky Slurm only (ci-runner queue) |
| Definition | .buildkite/pipeline.yml |
| Entrypoint | /opt/fastvideo-ci-runner/run-ci (run-unit is a compatibility wrapper) |
Fastcheck is the universal six-lane baseline. The merge gate does not repeat those jobs: it classifies every changed path and adds only the relevant lanes from the fourteen-lane integration set below. Selected jobs are hard gates; there are no soft-fail hardware lanes. A documentation-only PR can therefore finish its merge build after the trusted uploader, while model-family changes typically add focused golden-gate and SSIM files and a training-only change adds only its owning training lane.
.github/scripts/plan_merge_ci.py is the canonical path policy. It runs from
the immutable base SHA under pull_request_target; PR code is never executed
on the GitHub runner. The changed-file list includes both sides of renames. An
API failure, truncated response, empty list, unknown build input, or unknown
source path fails closed to all fourteen integration lanes.
A ready-labeled PR does not hit Buildkite immediately:
ci-trigger-full-suite.yml first runs .github/scripts/gate_full_suite.sh,
which waits for the cheap Tier-1 checks (pre-commit, docs build) on the PR
head. A red cheap check blocks the suite (fail closed; the next push re-arms
it), while a GitHub outage or a >25 min wait lets it run anyway (fail open).
/test full bypasses the gate.
The complete static graph remains available through /test full; path
selection never deletes or dynamically invents a Buildkite step.
Integration steps depend on golden-gate. A failed selected golden prevents
the expensive downstream jobs from starting; /test full still selects all
twenty lanes. A condition-skipped golden satisfies the dependency, so direct
lane reruns, scheduled SSIM, and training-only merge plans keep their existing
meaning. This follows Buildkite's
conditional dependency rules.
No dependency allows failures. The trusted uploader accepts either the old
complete graph or the complete golden-first graph during rollout, and rejects
partial or arbitrary dependency changes.
The six automatic Fastcheck lanes are unchanged. Within VAE and transformer lanes, small Wan goldens run before independent component parity. Wan paths select matching VAE, dense/trajectory, or causal-cache goldens; shared Wan config and pipeline wiring select all four. Shared runtime changes continue to select broader coverage. The existing block references retain their exact environment identity; new tensor gates distinguish the effective FA2/FA4 switch and VAE gates do not depend on an unused attention backend.
| Lane | Public TEST_TYPE |
GPUs | Typical merge trigger |
|---|---|---|---|
| Encoder | encoder |
1 | Universal Fastcheck |
| VAE | vae |
1 | Universal Fastcheck |
| Transformer | transformer |
1 | Universal Fastcheck |
| Kernel | kernel_tests |
1 | Universal Fastcheck |
| Unit | unit_test |
1 | Universal Fastcheck |
| DreamVerse | dreamverse_app |
1 | Universal Fastcheck |
| Golden gate | golden_gate |
1 | Model, pipeline, attention, layer, or output changes |
| SSIM | ssim |
4 | Matching model/SSIM paths; focused files when possible |
| LoRA inference | inference_lora |
1 | LoRA inference/shared LoRA paths |
| LoRA extraction | lora_extraction |
1 | LoRA extraction/shared LoRA paths |
| Vanilla training | training |
4 | Legacy vanilla/shared training paths |
| DMD distillation | distillation_dmd |
2 | DMD/shared training paths |
| Self-forcing | self_forcing |
2 | Self-forcing/shared training paths |
| LoRA training | training_lora |
2 | LoRA/shared training paths |
| VSA training | training_vsa |
2 | VSA/shared training paths |
| VMoBA inference | inference_vmoba |
1 | VMoBA backend/config paths |
| Performance | performance |
2 | Performance tests or benchmark policy |
| API server | api_server |
1 | API, worker, or server entrypoint paths |
| Modular train framework | train_framework |
1 | fastvideo/train/ and its tests |
| Eval metrics | eval |
1 | fastvideo/eval/ and its tests |
Golden-gate and SSIM selections are basenames, not arbitrary pytest arguments.
The private host checks the comma-separated allowlist before staging, and the
container checks it again before invoking pytest. Shared quality-harness
changes still run the complete owning lane. The full SSIM matrix also runs on
main every Sunday at 05:00 UTC through ci-scheduled-ssim.yml; /test ssim
and /test full remain available for deliberate complete runs.
The four Buildkite workers may accept multiple jobs concurrently. The
agent-owned lease broker packs their requested GPU counts onto one persistent
four-GPU Slurm allocation and waits when capacity is full. A four-GPU lane
such as SSIM or vanilla training owns the whole tray; two two-GPU lanes or up
to four one-GPU lanes can overlap without sharing devices. SSIM and vanilla
training also share the fastvideo/slinky/whole-tray Buildkite concurrency
group. That keeps the second whole-tray lane in Buildkite instead of consuming
an agent and its command timeout while the first lane waits for four free GPUs.
Because packed Enroot containers share the node network namespace, each GPU
lease receives its own 100-port rendezvous range. Tests preserve the
runner-assigned MASTER_PORT, and parallel SSIM tasks use distinct offsets
inside that range.
See Performance Benchmarks for the performance lane's thresholds, rolling baseline, artifacts, and reseeding process.
ci-scheduled-performance.yml triggers the performance pipeline every Sunday
at 23:00 America/Los_Angeles, passing the immutable github.sha. Its two UTC
schedules select the matching Pacific offset so it runs once across daylight
saving time. After this workflow lands, disable the legacy Buildkite schedule
under Settings → Schedules → FastVideo scheduled performance benchmarks →
Edit → Enabled. That schedule submits the literal HEAD, which the trusted
runner rejects before upload. The GitHub BUILDKITE_API_TOKEN must have
write_builds access to the performance pipeline.
The performance lane recognizes main-branch API builds from the dedicated
performance pipeline with TEST_SCOPE=direct as scheduled-main runs. They
refresh eligible baselines after successful tests and still produce comparison
reports after test failures. PR performance runs use strict baseline reads and
keep reports as Buildkite artifacts without requiring upload credentials.
Scheduled-main publishing requires an operator-provisioned token with write
access to FastVideo/performance-tracking.
The gated FLUX.1-dev SSIM and golden tests and FLUX.2-klein-9B SSIM test are explicitly paused until their model access is provisioned. Their pytest skips are not FLUX quality validation; the other SSIM and golden tests and the suites' fail-fast behavior remain active. The SD3.5 golden and SSIM tests are also explicitly paused at the maintainer's request while their gated checkpoint access is unavailable.
MatrixGame 3 explicitly selects FA2 for its existing GB200 reference. That
reference was published in dataset commit e1a6e64 and was not part of the
later FA4 refresh. Unchanged main with FA2 reproduces its video SHA-256
1d1d26a59481e37a9e838a628d875e196211c2236e8f9a370e0a9f73e97300c7;
FA4 scores 0.96636 against its unchanged 0.98 requirement. The test compares
the current generation's returned path so older files cannot mask that check.
The missing GB200 HunyuanVideo 1.5 I2V and Kandinsky 6 references were generated
from unchanged upstream commit d5253fd71055927f12aa6c4aa679b93a00238851
with Torch 2.12.0, CUDA 13 and FA4. Their checkpoint revisions are pinned in
the tests and recorded alongside the videos. Independent upstream repeats and
the candidate both scored 1.0 for Hunyuan and 0.98395 for Kandinsky against
the first upstream outputs, preserving the existing 0.95 and 0.98 thresholds.
Kandinsky remains required; its authorized complete snapshot is prepared in
the shared CI cache without exposing an authentication token to PR code.
LingBot Fast's missing GB200 reference uses the same upstream commit and
runtime, with checkpoint revision 98583eb0fd66f481f73383428d65d2867c878e51.
Two unchanged-upstream runs with CPU offloading disabled produced identical
video hashes. Its fixed FSDP CPU-offloaded run scored SSIM 1.0 against that
reference, preserving the 0.70 requirement. The generation stage leaves FSDP
parameter placement to FSDP instead of moving the whole model to the GPU.
Provisioning Required Hugging Face Models
Gated access belongs to the individual HF account, not its organization or token. An operator must request and obtain model access for the CI identity before configuring authentication. A valid token without model access still returns 403. Keep required models such as Kandinsky 6 strict until their weights and quality references are provisioned.
The permanent operator rollout is:
- Approve every required gated repository for one maintained CI identity.
- Configure HF CI/CD access through OIDC for the trusted Buildkite preparation step, restricting issuer and job claims to the intended organization and pipelines. A fine-grained read-only token in the operator's secret store is an alternative. Keep performance publishing credentials separate and restricted to the tracking dataset and trusted main runs.
- Before leasing GPUs, check access to all required pinned manifests and weight files together and check that their device/backend references exist. Report every missing dependency in one preflight result; do not turn access failures into passing quality gates.
- Fetch immutable model revisions into the runner-owned cache through trusted
code. Remove authentication before starting PR code, and keep download locks
and cache writes under the runner's identity. PR jobs consume the prepared
snapshots rather than depending on an interactive login or another user's
writable cache.
When multiple operators prepare the shared cache, use its existing group
with
umask 0002. Check group write access on each prepared model's cache, snapshot, blob and.locksdirectories, not only the cache root. Verify manifest resolution as the CI worker identity before running the lane. A readable weight file alone is insufficient: Hub downloads also create metadata and lock files. For example, Stable Audio's pinned revision is26609645c363fc81829282fcfe6356340346c1d4; provision its complete snapshot, includingmodel_index.json, component configs and tokenizer files. - For a missing SSIM reference, generate and inspect output from unchanged upstream source on the CI GPU/backend/runtime, retain its provenance, and publish only the reviewed new reference. Verify the candidate independently before enabling the required gate; never use candidate output to clear a regression failure.
See HF gated-model access and the testing guide for access and reference maintenance details.
/merge Request Flow
The Buildkite agent and Slurm have deliberately separate responsibilities:
/merge PR comment
-> GitHub verifies write permission, adds the `ready` label, and directly
calls the trusted base-branch merge-gate workflow
-> base-branch `ci-trigger-full-suite` workflow fetches the PR file list,
computes MERGE_TEST_PLAN plus focused golden/SSIM basenames, and gates on
cheap checks
-> sends the PR SHA to pipeline `ci` with TEST_SCOPE=merge and FULL_SUITE=true
-> trusted `pipeline-upload` job on queue `ci-runner`
fetch exact-SHA .buildkite/pipeline.yml
normalize + validate the complete static 20-lane policy and conditions
upload static Buildkite steps
-> each accepted step reaches the host policy hook
validate org/repo/SHA/ref/step key/command/scope/timeout
skip checkout on the login plane
stage a mode-0600 request on Lustre
lease 1-4 GPUs and attach an `srun` step to the Slinky tray
-> Enroot worker container
clone and verify the exact PR SHA
install the lane's project extras and cached kernel
run the repository-owned lane script
write numeric exit status and approved artifacts
-> trusted host returns that status to Buildkite
-> Buildkite publishes `full-suite-passed` only when every selected lane passes
The pipeline uploader and dispatcher run on the Slurm login plane, but those are control-plane operations only. Python tests, model loading, CUDA kernels, Node/Playwright checks, inference, training, SSIM generation, and performance benchmarks all execute in Slurm allocations.
PR-controlled values never become host commands. The host policy accepts only
the pinned pipeline uploader or a known lane tuple. It rejects plugins,
artifact globs, shell injection variables, non-immutable commits, and unknown
commands before checkout. Hugging Face credentials are added only for lanes
that declare them, passed through a mode-0600 request file, and removed before
the PR payload starts. Active training lanes keep W&B offline and do not stage
a W&B credential. The ARM64 image includes the pinned FA4 CuTe overlay validated
on GB200. SSIM opts into FA4 to preserve its reference-video numerics; lanes
with FA2 baselines keep FASTVIDEO_FA4=0. Performance artifacts are relayed
afterward by the trusted host from an allowlisted directory and extension set.
Slash Commands
Slash commands are handled by .github/workflows/ci-slash-commands.yml.
Repository write permission is required.
| Command | Effect |
|---|---|
/merge |
Adds ready and triggers the path-aware merge gate for the PR head. |
/test full |
Runs the whole Full Suite with TEST_SCOPE=full. |
/test fastcheck |
Runs the whole Fastcheck suite with TEST_SCOPE=fastcheck. |
/test pre-commit |
Re-runs the pre-commit workflow on the PR merge ref. |
/test <name> |
Runs one Buildkite test with TEST_SCOPE=direct. |
Valid direct test names:
| Command | TEST_TYPE |
|---|---|
/test encoder |
encoder |
/test vae |
vae |
/test transformer |
transformer |
/test kernel |
kernel_tests |
/test unit |
unit_test |
/test dreamverse |
dreamverse_app |
/test ssim |
ssim |
/test golden-gate |
golden_gate |
/test training |
training |
/test lora-inference |
inference_lora |
/test lora-training |
training_lora |
/test lora-extraction |
lora_extraction |
/test distillation |
distillation_dmd |
/test self-forcing |
self_forcing |
/test vsa |
training_vsa |
/test vmoba |
inference_vmoba |
/test performance |
performance |
/test api |
api_server |
/test train-framework |
train_framework |
/test eval |
eval |
The temporary <name>-ci spellings remain accepted as compatibility aliases;
they select the same Slurm lane and do not identify a second backend.
When a direct test completes successfully, Buildkite posts
direct-test-completed. .github/workflows/ci-aggregate-status.yml then reads
the latest Buildkite statuses for the commit and repairs an already-failed
fastcheck-passed or full-suite-passed status if all jobs in that group are
now green. A direct test never creates an aggregate for a tier that did not run.
Buildkite label emojis define the status namespace used by that aggregation:
:microscope: is reserved for the six Fastcheck lanes, while Full-Suite-only
lanes use :test_tube: or :bar_chart:. Each active lane has exactly one
label and therefore one status context.
Merge Protection
Mergify enforces these conditions before it squash-merges to main:
| Condition | Meaning |
|---|---|
check-success~=pre-commit |
Tier 1 passed. |
check-success=fastcheck-passed |
All triggered Fastcheck jobs passed. |
check-success=full-suite-passed |
The selected merge gate or explicit Full Suite passed. |
#approved-reviews-by>=1 |
At least one approving review. |
| Valid title regex | PR title starts with an accepted [type] tag. |
label=ready |
The PR has entered the merge flow. |
-draft |
PR is not a draft. |
-conflict |
PR has no merge conflicts. |
-closed |
PR is still open. |
The final merge action is a squash merge. Mergify also labels conflicting PRs
with needs-rebase and removes that label after conflicts are resolved.
Title Tags And Labels
PR title tags are the source for type:* labels and are required by merge
protection.
| Tag | Label | Use for |
|---|---|---|
[feat], [feature] |
type: feat |
New feature or capability |
[bugfix], [fix] |
type: bugfix |
Bug fix |
[refactor] |
type: refactor |
Code restructuring without behavior change |
[perf] |
type: perf |
Performance improvement |
[ci] |
type: ci |
CI/CD or build tooling changes |
[infra] |
type: infra |
Repo infrastructure, agent tooling, debug hooks, conversion scripts, dev infra |
[doc], [docs] |
type: docs |
Documentation only |
[misc], [chore] |
type: misc |
Housekeeping, dependency bumps, cleanup |
[kernel] |
No dedicated type label currently | CUDA kernel changes in fastvideo-kernel/ |
[new-model] |
type: new-model |
Adding a new model or pipeline |
[skill], [skills] |
type: skill |
Agent skills under .agents/skills/ or .claude/skills/ |
Scope labels are inferred from changed files.
| Label | File paths that trigger it |
|---|---|
scope: training |
fastvideo/train/, fastvideo/training/, fastvideo/distillation/, examples/train/, examples/training/, examples/distill/ |
scope: inference |
fastvideo/pipelines/basic/, fastvideo/pipelines/stages/, fastvideo/pipelines/samplers/, fastvideo/entrypoints/, fastvideo/worker/, fastvideo/api/sampling_param.py, fastvideo/configs/pipelines/, examples/inference/ |
scope: attention |
fastvideo/attention/ |
scope: kernel |
fastvideo-kernel/, csrc/ |
scope: data |
fastvideo/dataset/, fastvideo/pipelines/preprocess/, examples/preprocessing/ |
scope: infra |
.github/, .buildkite/, fastvideo/tests/, docker/ |
scope: distributed |
fastvideo/distributed/ |
scope: docs |
docs/ |
scope: studio |
apps/fastvideo_studio/ |
scope: model |
fastvideo/models/, fastvideo/layers/, fastvideo/configs/models/ |
Process labels:
| Label | Who sets it | Meaning |
|---|---|---|
ready |
/merge or maintainer action |
Triggers/keeps the change-aware merge gate active and enables auto-merge. |
needs-rebase |
Mergify | PR has merge conflicts. |
do-not-merge |
Maintainer | Blocks merge regardless of CI status. |
Slurm Lane Entrypoints
Every active test selection lives in .buildkite/scripts/unit_test.sh or a
focused .buildkite/scripts/lanes/<lane>.sh. The private, agent-owned lane
table binds each internal *_ci type to that script, its GPU count, wall-clock
limit, dependency extras, kernel-build policy, secrets, and artifacts. The
internal suffix is an implementation detail; there is only one active backend.
SSIM uses fastvideo/tests/ssim/ci_runner.py inside a single four-GPU lease.
It discovers REQUIRED_GPUS and *_MODEL_TO_PARAMS with AST parsing, then
packs independent pytest subprocesses across the visible GPUs with fail-fast
termination. Performance writes reports to a host-mounted artifact directory;
the host uploads only .md, .html, .json, and .csv files after the
container exits.
The Modal launchers remain in the repository for manual rollback archaeology,
but they are not CI entrypoints. pr_test.sh rejects Buildkite calls and needs
FASTVIDEO_ENABLE_LEGACY_MODAL_CI=1 even for a local manual invocation.
If you add a new CI test category:
- Add an executable
.buildkite/scripts/lanes/<lane>.shcontaining the test payload only. - Add the static Buildkite step in
.buildkite/pipeline.yml, its source/test ownership in.github/scripts/plan_merge_ci.py, and the/testmapping in.github/workflows/ci-slash-commands.yml. - Extend
fastvideo/tests/contract/test_ci_test_collection.py,test_merge_ci_plan.py, and the trusted private runner's lane table plus uploader policy in the same rollout. - Validate on the target GB200 hardware before making the lane a merge gate.
- Document the lane here and link any domain-specific authoring guide.
CD And Release Workflows
Documentation
.github/workflows/infra-docs.yml validates documentation changes on PRs and
deploys to GitHub Pages on pushes to main. Its path filters cover documentation,
examples and related build tooling; see the workflow for the exact triggers.
The docs job:
- Installs Python and Node documentation dependencies.
- Runs lint, formatting checks and tests for documentation tooling.
- Generates required static assets.
- Builds the Blume site, validates routes and assets, and tests navigation and cookbook interactions.
- Uploads
docs-site/dist/as the Pages artifact and publishes only frommain, at the existinghttps://haoailab.com/FastVideo/URL.
Docker Images
.github/workflows/infra-build-image.yml supports manual workflow_dispatch
runs and automatically rebuilds the CUDA matrix when a repository-controlled
image input changes on main in the canonical repository. Those inputs include
the CUDA Dockerfile and reusable workflow, dependency metadata, Docker context
policy, fastvideo-kernel/**, and the kernel artifact metadata/key helper.
Manual runs let maintainers choose which image families to build. The
fastvideo-dev matrix builds Python 3.12 images for CUDA 12.6 and CUDA 13 on
native amd64 and arm64 runners, then publishes one multi-platform manifest
per CUDA version. CUDA 12.6 owns the py3.12-latest and global latest tags, as
well as the explicit py3.12-cuda12.6.3-latest alias. CUDA 13 is published under
the explicit py3.12-cuda13.0.0-latest tag. This publication policy does not
change the unparameterized docker/Dockerfile build defaults, which remain CUDA
13 and cu130.
Published development images carry architecture-specific kernel wheels under
/opt/fastvideo-kernel-prebuilt. The Slurm worker selects the exact source,
ABI, and GPU-architecture match, so normal lanes reuse the trusted artifact
while kernel-changing PRs still build locally. Once a kernel or artifact-key
change reaches main, the image workflow republishes the matching artifact
before later jobs consume the updated image pin.
The same workflow publishes a single-architecture ARM64, CUDA 13, SM100 image
for the self-hosted CI runner under the
py3.12-cuda13.0.0-sm100-{latest,sha-*} tags. It carries the matching prebuilt
kernel wheel so runner jobs can validate and install the exact source and ABI
match instead of recompiling it in every lane.
The optional Dreamverse matrix builds backend and UI images for CUDA 12.6 and
CUDA 13 on amd64. Dreamverse remains amd64-only because its FA4 dependency
stack is not yet validated on ARM64.
The reusable implementation lives in
.github/workflows/_template-build-image.yml.
Package Publishing
| Workflow | Trigger | Publishes |
|---|---|---|
publish-fastvideo.yml |
pyproject.toml changes on main, or manual dispatch |
fastvideo package to PyPI when the version changes |
publish-kernel.yml |
fastvideo-kernel/pyproject.toml changes on main, or manual dispatch |
fastvideo-kernel wheels to PyPI when the version changes |
publish-comfyui.yml |
pyproject.toml changes on main/master, or manual dispatch |
ComfyUI custom node to the Comfy registry |
Community Automations
| Workflow | Trigger | Purpose |
|---|---|---|
community-issue-labeler.yml |
Issue opened or edited | Adds scope/platform labels from issue keywords. |
None of these automations post comments or messages on issues or pull requests; they only set labels. Issues and pull requests are never marked stale or closed automatically. Keep it that way: new automation must report through checks, commit statuses, or labels instead of PR/issue comments.
File Reference
| File | Owner area |
|---|---|
.github/mergify.yml |
Merge protection, PR title validation, PR labels, conflict labels, auto-merge |
.github/workflows/ci-precommit.yml |
Tier 1 pre-commit |
.github/workflows/ci-slash-commands.yml |
/merge and /test handling |
.github/workflows/ci-trigger-full-suite.yml |
Change-aware merge-gate trigger for ready PRs and new pushes |
.github/workflows/ci-scheduled-ssim.yml |
Weekly complete SSIM trigger on main |
.github/workflows/ci-scheduled-performance.yml |
Weekly performance trigger with an immutable source SHA |
.github/scripts/plan_merge_ci.py |
Trusted changed-path to integration-lane planner |
.github/workflows/ci-aggregate-status.yml |
Aggregate Fastcheck and explicit Full Suite direct-rerun statuses |
.buildkite/pipeline.yml |
Static 20-lane Slurm Buildkite graph |
.buildkite/scripts/unit_test.sh, .buildkite/scripts/lanes/*.sh |
Active Slurm lane payloads |
fastvideo/tests/ssim/ci_runner.py |
Four-GPU Slurm SSIM scheduler |
.buildkite/scripts/pr_test.sh, fastvideo/tests/modal/*.py |
Dormant manual Modal rollback path (disabled in Buildkite) |
.buildkite/performance-benchmarks/tests/*.json |
Performance benchmark configs and thresholds |
.github/workflows/infra-docs.yml |
Docs build and GitHub Pages deploy |
.github/workflows/infra-build-image.yml |
CUDA matrix, CI runner image, and manual Docker image builds |
.github/workflows/publish-fastvideo.yml |
FastVideo PyPI publishing |
.github/workflows/publish-kernel.yml |
FastVideo kernel PyPI publishing |
.github/workflows/publish-comfyui.yml |
ComfyUI registry publishing |