diff --git a/README.md b/README.md new file mode 100644 index 0000000..51b23a6 --- /dev/null +++ b/README.md @@ -0,0 +1,124 @@ +# ComfyUI-RIFE + +English | [中文](README_zh.md) + +RIFE video frame interpolation node for ComfyUI, based on Practical-RIFE implementation. + +## Features + +- **Real-time Video Frame Interpolation**: Uses RIFE (Real-Time Intermediate Flow Estimation) algorithm to generate smooth intermediate frames between video frames +- **Automatic Model Download**: Automatically downloads required models from GitHub Releases on first use +- **Model Caching**: Automatically caches loaded models to avoid reloading +- **Multiple Precision Support**: Supports float32, float16, and bfloat16 precision for speed/quality trade-off +- **Torch Compile**: Optional PyTorch 2.0+ `torch.compile()` acceleration (slower first run, faster subsequent inference) +- **Batch Inference**: Supports batch processing of interpolation tasks for higher throughput + +## Supported Models + +| Model Name | Version | Description | +|-----------|---------|-------------| +| `rife47.pth` | 4.7 | RIFE version 4.7 | +| `rife49.pth` | 4.7 | RIFE version 4.9 | +| `rife417.pth` | 4.17 | RIFE version 4.17 | +| `rife426.pth` | 4.26 | RIFE version 4.26 (ensemble not supported) | + +Models are automatically downloaded to `ComfyUI/models/rife/` directory. + +## Installation + +1. Clone this repository into ComfyUI's `custom_nodes` directory: + ```bash + cd ComfyUI/custom_nodes + git clone ComfyUI-RIFE + ``` + +2. Install dependencies: + ```bash + cd ComfyUI-RIFE + pip install -r requirements.txt + ``` + +## Node Parameters + +### RIFE_VFI + +**Required Parameters:** + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `ckpt_name` | Dropdown | - | Select RIFE model (auto-downloads if not present) | +| `frames` | IMAGE | - | Input video frame sequence | +| `clear_cache_after_n_frames` | INT | 10 | Clear GPU cache every N frames (1-1000) | +| `multiplier` | INT | 2 | Interpolation multiplier, e.g., 2 generates 1 intermediate frame between each pair | +| `fast_mode` | BOOLEAN | True | Fast mode, trade some quality for speed | +| `ensemble` | BOOLEAN | True | Use ensemble mode for better quality (forced off for v4.26) | +| `scale_factor` | Dropdown | 1.0 | Optical flow scale factor [0.25, 0.5, 1.0, 2.0, 4.0] | +| `dtype` | Dropdown | float32 | Inference precision: float32 (safest), float16 (fast/less VRAM), bfloat16 (requires RTX 30xx+) | +| `torch_compile` | BOOLEAN | False | Use torch.compile() for acceleration (requires PyTorch 2.0+) | +| `batch_size` | INT | 1 | Number of interpolation tasks per batch (higher = more throughput but more VRAM, 1-64) | + +**Optional Parameters:** + +| Parameter | Type | Description | +|-----------|------|-------------| +| `optional_interpolation_states` | INTERPOLATION_STATES | Interpolation state list for controlling frame skipping | + +**Outputs:** + +| Output | Type | Description | +|--------|------|-------------| +| `IMAGE` | IMAGE | Interpolated video frame sequence | + +## Usage Example + +### Basic Workflow + +1. Load video frames using `Load Video` or `Load Image Batch` node +2. Connect to `RIFE_VFI` node +3. Select a model from the dropdown (e.g., `rife426.pth`) +4. Set `multiplier` (e.g., 2 means 30fps → 60fps) +5. Output using `Save Video` or `Preview Image` node + +### Parameter Tuning Suggestions + +- **Quality Priority**: `dtype=float32`, `ensemble=True`, `fast_mode=False` +- **Speed Priority**: `dtype=float16`, `fast_mode=True`, `torch_compile=True` (PyTorch 2.0+) +- **Low VRAM**: Reduce `batch_size` to 1, use `dtype=float16` +- **RTX 30xx/40xx**: Can use `dtype=bfloat16`, more stable than float16 with similar speed + +## File Structure + +``` +ComfyUI-RIFE/ +├── __init__.py # Node registration entry +├── rife_node.py # Main node implementation (model loading, inference) +├── rife_model.py # RIFE IFNet model definition +├── requirements.txt # Python dependencies +└── README.md # This file (Chinese) +└── README_EN.md # English version +``` + +## Model Download Notes + +Model files are automatically downloaded to `ComfyUI/models/rife/`: + +- On first use of a model, the node automatically downloads from `https://github.com/Fannovel16/ComfyUI-Frame-Interpolation/releases/download/models/` +- If download fails, check network connection or manually download models to the above directory + +## Important Notes + +1. **RIFE 4.26**: This version does not support `ensemble` mode, it will be automatically disabled +2. **torch.compile()**: First inference will be slow (compilation phase), subsequent runs are 10-30% faster +3. **bfloat16**: Only supported on Ampere architecture (RTX 30xx/40xx) or newer GPUs +4. **Model Caching**: Changing `ckpt_name`, `dtype`, or `torch_compile` triggers model reload + +## Dependencies + +- torch >= 2.0.0 (recommended) +- einops +- packaging +- numpy + +## License + +This project is based on Practical-RIFE implementation, following the corresponding open-source license. diff --git a/README_zh.md b/README_zh.md new file mode 100644 index 0000000..bbff21a --- /dev/null +++ b/README_zh.md @@ -0,0 +1,123 @@ +# ComfyUI-RIFE + +[English](README.md) | 中文 + +用于 ComfyUI 的 RIFE 视频插帧节点,基于 Practical-RIFE 实现。 + +## 功能特性 + +- **实时视频插帧**: 使用 RIFE(Real-Time Intermediate Flow Estimation)算法在视频帧之间生成平滑的中间帧 +- **自动模型下载**: 首次使用时会自动从 GitHub Releases 下载所需模型 +- **模型缓存**: 自动缓存已加载的模型,避免重复加载 +- **多种精度支持**: 支持 float32、float16、bfloat16 精度,可在速度和精度间权衡 +- **Torch Compile**: 可选 PyTorch 2.0+ 的 `torch.compile()` 加速(首次编译较慢,后续推理更快) +- **批量推理**: 支持批量处理插帧任务,提高吞吐量 + +## 支持的模型 + +| 模型名称 | 版本 | 说明 | +|---------|------|------| +| `rife47.pth` | 4.7 | RIFE 4.7 版本 | +| `rife49.pth` | 4.7 | RIFE 4.9 版本 | +| `rife417.pth` | 4.17 | RIFE 4.17 版本 | +| `rife426.pth` | 4.26 | RIFE 4.26 版本(不支持 ensemble)| + +模型会自动下载到 `ComfyUI/models/rife/` 目录。 + +## 安装 + +1. 将本节点文件夹放入 ComfyUI 的 `custom_nodes` 目录: + ```bash + cd ComfyUI/custom_nodes + git clone ComfyUI-RIFE + ``` + +2. 安装依赖: + ```bash + cd ComfyUI-RIFE + pip install -r requirements.txt + ``` + +## 节点参数 + +### RIFE_VFI + +**必需参数:** + +| 参数名 | 类型 | 默认值 | 说明 | +|--------|------|--------|------| +| `ckpt_name` | 下拉选择 | - | 选择 RIFE 模型(自动下载) | +| `frames` | IMAGE | - | 输入视频帧序列 | +| `clear_cache_after_n_frames` | INT | 10 | 每处理 N 帧后清理显存缓存(1-1000) | +| `multiplier` | INT | 2 | 插帧倍数,如 2 表示在每帧之间生成 1 个中间帧 | +| `fast_mode` | BOOLEAN | True | 快速模式,牺牲少量质量换取速度 | +| `ensemble` | BOOLEAN | True | 使用 ensemble 模式提高质量(4.26 版本强制关闭) | +| `scale_factor` | 下拉选择 | 1.0 | 光流计算缩放比例 [0.25, 0.5, 1.0, 2.0, 4.0] | +| `dtype` | 下拉选择 | float32 | 推理精度:float32(最安全)、float16(快/省显存)、bfloat16(需 RTX 30xx+) | +| `torch_compile` | BOOLEAN | False | 使用 torch.compile() 加速(需 PyTorch 2.0+) | +| `batch_size` | INT | 1 | 每批次处理的插帧任务数(提高吞吐量但增加显存占用,1-64) | + +**可选参数:** + +| 参数名 | 类型 | 说明 | +|--------|------|------| +| `optional_interpolation_states` | INTERPOLATION_STATES | 插帧状态列表,用于控制跳过特定帧 | + +**输出:** + +| 输出名 | 类型 | 说明 | +|--------|------|------| +| `IMAGE` | IMAGE | 插帧后的视频帧序列 | + +## 使用示例 + +### 基本流程 + +1. 使用 `Load Video` 或 `Load Image Batch` 节点加载视频帧 +2. 连接到 `RIFE_VFI` 节点 +3. 从下拉菜单选择模型(如 `rife426.pth`) +4. 设置 `multiplier`(如 2 表示 30fps → 60fps) +5. 使用 `Save Video` 或 `Preview Image` 节点输出结果 + +### 参数调优建议 + +- **追求质量**: `dtype=float32`, `ensemble=True`, `fast_mode=False` +- **追求速度**: `dtype=float16`, `fast_mode=True`, `torch_compile=True`(PyTorch 2.0+) +- **显存不足**: 降低 `batch_size` 至 1,使用 `dtype=float16` +- **RTX 30xx/40xx**: 可尝试 `dtype=bfloat16`,比 float16 更稳定且速度相近 + +## 文件结构 + +``` +ComfyUI-RIFE/ +├── __init__.py # 节点注册入口 +├── rife_node.py # 节点主要实现(模型加载、推理逻辑) +├── rife_model.py # RIFE IFNet 模型定义 +├── requirements.txt # Python 依赖 +└── README.md # 本文件 +``` + +## 模型下载说明 + +模型文件会自动下载到 `ComfyUI/models/rife/` 目录: + +- 首次使用某个模型时,节点会自动从 `https://github.com/Fannovel16/ComfyUI-Frame-Interpolation/releases/download/models/` 下载 +- 如果下载失败,请检查网络连接或手动下载模型放到上述目录 + +## 注意事项 + +1. **RIFE 4.26 版本**: 该版本不支持 `ensemble` 模式,会自动强制关闭 +2. **torch.compile()**: 首次推理会很慢(编译阶段),后续推理会快 10-30% +3. **bfloat16**: 仅在 Ampere 架构(RTX 30xx/40xx)或更新的 GPU 上支持 +4. **模型缓存**: 修改 `ckpt_name`、`dtype` 或 `torch_compile` 会触发重新加载模型 + +## 依赖 + +- torch >= 2.0.0(推荐) +- einops +- packaging +- numpy + +## 许可证 + +本项目基于 Practical-RIFE 实现,遵循相应开源协议。