[feat] v2: port Wan2.1-T2V-14B (bucket B) + document the all-models backlog

First bucket-B port toward 'support every fastvideo model': Wan2.1-T2V-14B reuses the Wan recipe +
torch adapter unchanged (same WanTransformer3DModel/AutoencoderKLWan/UMT5) — only a registry entry +
build_wan_t2v_14b_card (720p, flow_shift 5.0) + SamplingDefaults differ. Without the entry the arch
fallback would give it the 1.3B 480p defaults; the explicit entry gives 50 steps / 720x1280.

Also recorded the full backlog in V2_PORTING_STATUS.md: 63 fastvideo models = 8 ported / 21 bucket-B
(reuse Wan/Causal/LTX2 arch — registry+recipe+defaults, no new adapter) / 34 bucket-C (13 new
architectures needing a TorchComponent adapter). Updated the stale 'how to add a model' steps to the
post-redesign structure (v2/recipes/, torch_backend.py, SamplingDefaults). CPU mini 211 pass, 2 skip.
This commit is contained in:
Will Lin
2026-07-04 17:15:34 +00:00
committed by Peiyuan Zhang
parent 440b99523e
commit 5f1d2ef7d2
5 changed files with 49 additions and 11 deletions
+19 -9
View File
@@ -54,8 +54,16 @@ Wan2.1 model — covered by wan21; those runtime modes are out of scope.
| `basic_lucy_edit.py` | `decart-ai/Lucy-Edit-Dev` (Wan v2v) | **V2V mechanism (identified):** `WanVideoToVideoPipeline` + `VideoVAEEncodingStage` — VAE-encode the input video to a conditioning latent. v2 needs input-video loading + a VAE-encode-video node + the v2v conditioning (no image_encoder). |
| `basic_ltx2_3_distilled_i2v*.py` | LTX-2.3-Distilled I2V | Image conditioning on the ltx2 single-stage program (encode the input image + condition the denoise). |
## Out of current scope — new families (kept as an upstream reference map only)
Each would need a new card + adapters (DiT/VAE/encoder forwards) + maybe a sampler:
## Backlog — goal: port ALL fastvideo models (63 total: 8 ported · 21 reuse-arch · 34 new-arch)
Ported (8): the 7 above + **Wan2.1-T2V-14B** (bucket B — reuses the Wan recipe/adapter; 720p defaults).
**Bucket B (21 left) — reuse a v2 architecture** (`WanTransformer3DModel` / `CausalWanTransformer3DModel` /
`LTX2Transformer3DModel`): a `ModelEntry` in `v2/registry.py` + a recipe (parameterize the wan21/wan_causal/
ltx2 builders) + `SamplingDefaults`, NO new adapter. Includes other Wan T2V sizes; i2v variants (need an
image-encoder component + i2v program); FastWan/Turbo (DMD/RCM schedules → a loop variant); Wan2.2-I2V-A14B;
SF-Wan2.2; LTX-2 base/2.3 repo aliases (already arch-fallback-resolvable, so they work without an entry).
**Bucket C (34 models, ~13 new architectures) — need a new `TorchComponent` adapter + recipe per arch:**
`basic_cosmos2_5_*` (Cosmos-Predict2.5), `basic_gen3c.py` (GEN3C — **local `converted_weights/` only**),
`basic_hy15*.py` / `basic_gamecraft.py` / `basic_hyworld.py` (Hunyuan family),
`basic_longcat_*.py` (LongCat + LoRA), `basic_lingbotworld_base_cam.py`,
@@ -66,11 +74,13 @@ Each would need a new card + adapters (DiT/VAE/encoder forwards) + maybe a sampl
(interactive mouse/keyboard-conditioned world models).
## How to add a model to the v2 VideoGenerator
1. Ensure a v2 card exists (`v2/models/<family>/`) whose `load_id`s resolve to real `fastvideo.models.*`.
2. Register it in the shared `v2/registry.py` (used by every entrypoint — VideoGenerator, CLI, server):
add a `ModelEntry` mapping its HF id(s) → builders (the explicit PRIMARY path — the only way to split
same-architecture capability variants, e.g. T2V vs the i2v "InP"), and/or a branch in
`select_by_architecture` for the arch-inference FALLBACK (local paths / unregistered repos).
3. If a DiT/VAE/encoder/upsampler forward differs, add an adapter + class-detection in `torch_adapters.py`
and register the component kind in `torch_cuda.py`.
1. Add a v2 recipe under `v2/recipes/<family>/` (card/loop/program) whose `load_id`s name the real models;
set per-model `SamplingDefaults` on the card (`v2/card/specs.py`) — see `wan21`/`ltx2` for the pattern.
2. Register it in the shared `v2/registry.py` (every entrypoint resolves through it): add a `ModelEntry`
mapping its HF id(s) → builders (the PRIMARY path — the only way to split same-arch capability variants,
e.g. T2V vs the i2v "InP"), and/or a branch in `select_by_architecture` for the arch-inference FALLBACK.
3. **Bucket B (reuse an existing arch):** nothing more — the redesigned `torch_backend.py` already builds
Wan/LTX components. **Bucket C (new arch):** add a `TorchComponent` subclass + a `_MAKERS`/class-detect
branch in `torch_backend.py`, vendor the new model code as marked re-export stubs under `v2/models/...`
(see `v2-vendoring-approach`), and register any new component kind in `torch_cuda.py`.
4. Add `examples/inference/basic/v2_<name>.py` mirroring the official example; generate to confirm.
+2 -1
View File
@@ -5,10 +5,11 @@ from v2.recipes.wan21.card import (
build_wan21_card,
build_wan22_a14b_card,
build_wan22_ti2v_card,
build_wan_t2v_14b_card,
stamp_wan21_checkpoints,
)
from v2.recipes.wan21.loop import WanDenoiseLoop, latent_shape
from v2.recipes.wan21.program import build_wan_t2v_program
__all__ = ["build_wan21_card", "build_wan22_ti2v_card", "build_wan22_a14b_card",
__all__ = ["build_wan21_card", "build_wan22_ti2v_card", "build_wan22_a14b_card", "build_wan_t2v_14b_card",
"stamp_wan21_checkpoints", "build_wan_t2v_program", "WanDenoiseLoop", "latent_shape"]
+10
View File
@@ -118,6 +118,16 @@ def build_wan22_ti2v_card(model_id: str = "wan2.2-ti2v-5b", *,
num_frames=121, fps=24, negative_prompt=WAN_NEG_CN))
def build_wan_t2v_14b_card(model_id: str = "wan2.1-t2v-14b", *, checkpoint_root: str | None = None) -> ModelCard:
"""Wan2.1-T2V-14B (720p). Same WanTransformer3DModel / AutoencoderKLWan / UMT5 + 16/8/4 VAE geometry
as the 1.3B — reuses the Wan recipe + torch adapter unchanged; only the size (resolved from the
checkpoint by the loader), the 720p flow shift 5.0, and the per-model defaults differ."""
return build_wan21_card(model_id=model_id, flow_shift=5.0, checkpoint_root=checkpoint_root,
sampling_defaults=SamplingDefaults(
num_steps=50, guidance_scale=5.0, height=720, width=1280,
num_frames=81, fps=16, negative_prompt=WAN_NEG_EN))
def build_wan22_a14b_card(model_id: str = "wan2.2-t2v-a14b", *, checkpoint_root: str | None = None,
boundary: float = 0.875) -> ModelCard:
"""Wan2.2-T2V-A14B card — MoE: two WanTransformer3DModel experts (in_ch=16, Wan2.1 16/8/4 geometry)
+2
View File
@@ -48,10 +48,12 @@ def _entries() -> list[ModelEntry]:
build_wan21_card,
build_wan22_a14b_card,
build_wan22_ti2v_card,
build_wan_t2v_14b_card,
build_wan_t2v_program,
)
return [
ModelEntry(("Wan-AI/Wan2.1-T2V-1.3B-Diffusers",), build_wan21_card, build_wan_t2v_program),
ModelEntry(("Wan-AI/Wan2.1-T2V-14B-Diffusers",), build_wan_t2v_14b_card, build_wan_t2v_program),
ModelEntry(("Wan-AI/Wan2.2-TI2V-5B-Diffusers",), build_wan22_ti2v_card, build_wan_t2v_program),
ModelEntry(("Wan-AI/Wan2.2-T2V-A14B-Diffusers",), build_wan22_a14b_card, build_wan_t2v_program),
ModelEntry(("wlsaidhi/SFWan2.1-T2V-1.3B-Diffusers",), build_wan_causal_card, build_wan_causal_program),
+16 -1
View File
@@ -3,11 +3,26 @@ from types import SimpleNamespace
from v2.card import SamplingDefaults
from v2.recipes.ltx2 import build_ltx2_3_card, build_ltx2_base_card, build_ltx2_card
from v2.recipes.wan21 import build_wan21_card, build_wan22_a14b_card, build_wan22_ti2v_card
from v2.recipes.wan21 import (
build_wan21_card,
build_wan22_a14b_card,
build_wan22_ti2v_card,
build_wan_t2v_14b_card,
)
from v2.recipes.wan_causal import build_wan_causal_card
from v2.registry import resolve
from v2.video_generator import _resolve_default
def test_wan_t2v_14b_defaults_and_registry():
# Bucket-B port: reuses the Wan recipe/adapter, only defaults differ (720p, flow_shift 5.0).
sd = build_wan_t2v_14b_card().sampling_defaults
assert (sd.num_steps, sd.guidance_scale, sd.height, sd.width, sd.fps) == (50, 5.0, 720, 1280, 16)
# the HF id resolves to the 14B card via the shared registry (not the 1.3B fallback defaults)
build_card, _ = resolve("Wan-AI/Wan2.1-T2V-14B-Diffusers")
assert build_card().sampling_defaults.height == 720
def test_wan21_defaults():
sd = build_wan21_card().sampling_defaults
assert (sd.num_steps, sd.guidance_scale, sd.height, sd.width, sd.num_frames, sd.fps) == (50, 3.0, 480, 832, 81, 16)