64 lines
2.1 KiB
Markdown
64 lines
2.1 KiB
Markdown
|
|
# Documentation
|
|
- Class name: MaskOptFlow
|
|
- Category: ControlNet Preprocessors/Optical Flow
|
|
- Output node: False
|
|
- Repo Ref: https://github.com/comfyanonymous/ComfyUI
|
|
|
|
MaskOptFlow节点用于对光流输入应用蒙版，有效地根据提供的蒙版过滤光流数据。这项操作对于从光流数据中特定感兴趣区域隔离相关运动信息至关重要。
|
|
|
|
# Input types
|
|
## Required
|
|
- optical_flow
|
|
- 光流输入代表两个连续帧之间的运动。它对于理解场景内的动态变化至关重要。
|
|
- Comfy dtype: OPTICAL_FLOW
|
|
- Python dtype: torch.Tensor
|
|
- mask
|
|
- 蒙版输入用于过滤光流数据，仅保留感兴趣区域的运动信息。
|
|
- Comfy dtype: MASK
|
|
- Python dtype: torch.Tensor
|
|
|
|
# Output types
|
|
- OPTICAL_FLOW
|
|
- 经过过滤的光流数据，其中只保留了蒙版指定区域的运动信息。
|
|
- Comfy dtype: OPTICAL_FLOW
|
|
- Python dtype: torch.Tensor
|
|
- PREVIEW_IMAGE
|
|
- 过滤后光流的可视化，提供蒙版保留的运动信息的视觉表示。
|
|
- Comfy dtype: IMAGE
|
|
- Python dtype: torch.Tensor
|
|
|
|
|
|
## Usage tips
|
|
- Infra type: `GPU`
|
|
- Common nodes: unknown
|
|
|
|
|
|
## Source code
|
|
```python
|
|
class MaskOptFlow:
|
|
@classmethod
|
|
def INPUT_TYPES(s):
|
|
return {
|
|
"required": dict(optical_flow=("OPTICAL_FLOW",), mask=("MASK",))
|
|
}
|
|
|
|
RETURN_TYPES = ("OPTICAL_FLOW", "IMAGE")
|
|
RETURN_NAMES = ("OPTICAL_FLOW", "PREVIEW_IMAGE")
|
|
FUNCTION = "mask_opt_flow"
|
|
|
|
CATEGORY = "ControlNet Preprocessors/Optical Flow"
|
|
|
|
def mask_opt_flow(self, optical_flow, mask):
|
|
from controlnet_aux.unimatch import flow_to_image
|
|
assert len(mask) >= len(optical_flow), f"Not enough masks to mask optical flow: {len(mask)} vs {len(optical_flow)}"
|
|
mask = mask[:optical_flow.shape[0]]
|
|
mask = F.interpolate(mask, optical_flow.shape[1:3])
|
|
mask = rearrange(mask, "n 1 h w -> n h w 1")
|
|
vis_flows = torch.stack([torch.from_numpy(flow_to_image(flow)).float() / 255. for flow in optical_flow.numpy()], dim=0)
|
|
vis_flows *= mask
|
|
optical_flow *= mask
|
|
return (optical_flow, vis_flows)
|
|
|
|
```
|