shaoxiongduan
f6275f8005
Adds an in-process evaluation suite covering native (SSIM/PSNR/LPIPS/ optical_flow), audio, physics_iq, vbench (16 sub-metrics), and videoscore2. Public API: create_evaluator/evaluate, BaseMetric + @register, ensure_checkpoint, get_cache_dir. CLI: fastvideo eval list/run. VBench upstream pinned as a git submodule at fastvideo/third_party/eval/vbench (Vchitect/VBench@45e79ec); modern-dep compat is achieved via runtime shims in vbench/__init__.py rather than on-disk patches. CLIP/torch.hub caches are routed through ${FASTVIDEO_EVAL_CACHE}; HF cache stays at the system default. ensure_checkpoint delegates to fastvideo.utils.get_lock + huggingface_hub primitives. Evaluator supports release_cuda_memory(), unload(), reload() for training-time eval that frees GPU between calls. Verified parity against upstream vbench (5/8 metrics bit-exact, others within 1% drift driven by transformers/torch version skew) and against upstream VideoScore2 (regex anchored on the actual model's output format, soft-score formula matches upstream's argmax*max_prob/total). Includes docs/contributing/eval-metrics.md as the porting guide. Out of scope (deferred follow-ups): MIND, VBench-2.0, FVD as a registered metric, training-time EvalCallback.
10 lines
405 B
Plaintext
10 lines
405 B
Plaintext
[submodule "fastvideo-kernel/include/tk"]
|
|
path = fastvideo-kernel/include/tk
|
|
url = https://github.com/HazyResearch/ThunderKittens.git
|
|
[submodule "fastvideo-kernel/include/cutlass"]
|
|
path = fastvideo-kernel/include/cutlass
|
|
url = https://github.com/NVIDIA/cutlass.git
|
|
[submodule "fastvideo/third_party/eval/vbench"]
|
|
path = fastvideo/third_party/eval/vbench
|
|
url = https://github.com/Vchitect/VBench.git
|