# ComfyUI KeySync Wrapper

This custom node package wraps the official KeySync inference pipeline for ComfyUI.

It does not include KeySync weights and it does not rewrite KeySync model code. The first version calls the official `scripts/infer_raw.sh` script through `subprocess`.

## Nodes

- `KeySync Download Models`
- `KeySync Model Loader`
- `KeySync LipSync`
- `KeySync LipSync Advanced`

Typical workflow:

```text
Load Video (Path) -> KeySync LipSync.images
Load Audio (Path) -> KeySync LipSync.audio
KeySync LipSync.images/audio -> VideoHelperSuite Video Combine
```

`KeySync LipSync` returns:

- `images`: ComfyUI `IMAGE` frames
- `audio`: ComfyUI `AUDIO`
- `frame_rate`: usually `25`
- `output_video_path`: path to the intermediate `final.mp4`

## Install The Official KeySync Repo

Clone the original KeySync repository into this wrapper folder:

```bash
cd D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper
git clone https://github.com/antonibigata/keysync
```

Default repo path used by the loader:

```text
D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper/keysync
```

## Download Models

Recommended model directory:

```text
D:/ComfyUI/models/keysync
```

You can use the `KeySync Download Models` node with `download=True`.

Manual download is also supported:

```bash
git lfs install
git clone https://huggingface.co/toninio19/keysync D:/ComfyUI/models/keysync
```

The loader expects:

```text
D:/ComfyUI/models/keysync/checkpoints/keyframe_dub.pt
D:/ComfyUI/models/keysync/checkpoints/interpolation_dub.pt
```

The default UI intentionally hides repo and model directory fields. The wrapper checks these fixed paths:

```text
KeySync repo: D:/ComfyUI/custom_nodes/ComfyUI-Keysync-wrapper/keysync
Model dir:    D:/ComfyUI/models/keysync
```

## Runtime Notes

KeySync has heavy dependencies. This wrapper intentionally does not install the full official KeySync dependency set into ComfyUI.

Recommended runtime:

- WSL2 or Linux
- `ffmpeg` available on `PATH`
- `bash` available on `PATH`
- official KeySync environment/dependencies installed according to the upstream README

If you run native Windows ComfyUI, install Git Bash or use WSL2. The official inference script depends on bash.

## Output Layout

Each run creates an isolated task folder:

```text
D:/ComfyUI/output/keysync/task_YYYYMMDD_HHMMSS_xxxxxxxx/
  data/
    videos/input.mp4
    audios/input.wav
  logs/
    keysync_stdout.log
    keysync_stderr.log
  final.mp4
```

The official KeySync repo also receives its normal output under:

```text
<keysync_repo_path>/outputs/<task_id>/
```

Set `keep_intermediate_outputs=False` to remove the prepared input data and upstream output folder after `final.mp4` is copied.

## Parameters

The simple `KeySync LipSync` node exposes only the everyday parameters:

- `input_fps`
- `compute_until`

Use `KeySync LipSync Advanced` only when you need the official occlusion path:

- `fix_occlusion`: enables KeySync's official SAM2-based occlusion mask path.
- `position`: point prompt for the occluding object, for example `450,450`.
- `start_frame`: frame index where the point prompt applies.

Internal wrapper behavior is fixed for the simple node:

- video is normalized to 25 fps
- audio is normalized to 16 kHz mono WAV
- subprocess mode is always used
- intermediate task files are kept for debugging
- no subprocess timeout is enforced by default

Advanced KeySync parameters such as `resize_size`, `chunk_size`, `cond_aug`, and `decoding_t` are not exposed in v1 because the official `infer_raw.sh` script currently hardcodes them.

## Occlusion And SAM

The official KeySync code uses SAM2 for occlusion handling. The `position` field is a SAM2 point prompt used to track the object that blocks the face, such as a hand or microphone. This is powerful but awkward as a text field, so it is hidden from the simple node.

SAM3-based tracking may be added later as a separate preprocessing node that outputs an occlusion mask or point prompt. It is not wired into v1 because the official KeySync `infer_raw.sh` path expects its own SAM2 implementation and checkpoint layout.

## Common Errors

Missing KeySync repo:

```text
KeySync repo path does not exist
```

Missing official script:

```text
scripts/infer_raw.sh not found
```

Missing checkpoints:

```text
keyframes_ckpt does not exist
interpolation_ckpt does not exist
```

Missing ffmpeg:

```text
ffmpeg not found. Please install ffmpeg and add it to PATH.
```

Missing bash:

```text
KeySync official inference script requires bash.
```

When KeySync fails, the error message includes the stdout and stderr log paths.