100 lines
4.0 KiB
Markdown
100 lines
4.0 KiB
Markdown
# video_to_text_cli.py
|
||
|
||
Submit a video to a remote ComfyUI server running `ComfyUI-VideoToText` + `ComfyUI-MiMoASR` and dump the JSON transcript locally.
|
||
|
||
## Install
|
||
|
||
```bash
|
||
uv pip install --system websocket-client requests
|
||
```
|
||
|
||
## Basic usage
|
||
|
||
```bash
|
||
# Upload a local file and transcribe with Auto language
|
||
uv run python video_to_text_cli.py /path/to/lecture.mp4 \
|
||
--ip 10.8.0.12 \
|
||
--model-path /big_disk/models/MiMo-V2.5-ASR \
|
||
--tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer
|
||
|
||
# Pure Chinese content
|
||
uv run python video_to_text_cli.py speech_zh.mp4 \
|
||
--ip 10.8.0.12 \
|
||
--model-path /big_disk/models/MiMo-V2.5-ASR \
|
||
--tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
|
||
--language Chinese
|
||
|
||
# Reference a file already in ComfyUI/input (no re-upload)
|
||
uv run python video_to_text_cli.py already_uploaded.mp4 \
|
||
--ip 10.8.0.12 \
|
||
--model-path /big_disk/models/MiMo-V2.5-ASR \
|
||
--tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer
|
||
|
||
# Dump workflow without running — feed into UI for debugging
|
||
uv run python video_to_text_cli.py video.mp4 \
|
||
--model-path /big_disk/models/MiMo-V2.5-ASR \
|
||
--tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
|
||
--dump-workflow /tmp/wf.json
|
||
```
|
||
|
||
## Skip retyping paths
|
||
|
||
Add an alias to your shell:
|
||
|
||
```bash
|
||
# ~/.bashrc
|
||
alias v2t='uv run python /path/to/ComfyUI-VideoToText/cli/video_to_text_cli.py \
|
||
--ip 10.8.0.12 \
|
||
--model-path /big_disk/models/MiMo-V2.5-ASR \
|
||
--tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
|
||
--repo-path /opt/repos/MiMo-V2.5-ASR'
|
||
|
||
# Then:
|
||
v2t my_video.mp4 --language Auto
|
||
v2t lecture.mp4 --language Chinese --out ./transcripts
|
||
```
|
||
|
||
## All flags
|
||
|
||
| Flag | Default | Purpose |
|
||
|---|---|---|
|
||
| `video` (positional) | — | Local path OR filename already in `ComfyUI/input`. |
|
||
| `--ip` | `127.0.0.1` | ComfyUI host. |
|
||
| `--port` | `8188` | ComfyUI port. |
|
||
| `--server` | — | Full `host:port` (overrides `--ip` / `--port`). |
|
||
| `--model-path` | **required** | Absolute path to `MiMo-V2.5-ASR` on the server. |
|
||
| `--tokenizer-path` | **required** | Absolute path to `MiMo-Audio-Tokenizer` on the server. |
|
||
| `--repo-path` | `""` | Absolute path to MiMo source repo. Empty if `MIMO_ASR_REPO` is set on the server. |
|
||
| `--force-reload` | off | Drop the MiMo loader cache and re-instantiate. |
|
||
| `--language` | `Auto` | `Auto` / `Chinese` / `English`. |
|
||
| `--sample-rate` | `16000` | ffmpeg output rate before MiMo. |
|
||
| `--save-prefix` | `transcript` | Filename prefix on the server side. |
|
||
| `--out` | `./out` | Local directory to save the JSON. |
|
||
| `--dump-workflow` | — | Path to dump generated workflow JSON. |
|
||
| `--timeout` | `1800` | WebSocket wait timeout (sec). |
|
||
|
||
## How it works
|
||
|
||
1. Upload the local video to `ComfyUI/input/` via `POST /upload/image`.
|
||
2. Build the 9-node API workflow:
|
||
- `VTT_LoadVideo` → `VTT_ExtractAudio` → `VTT_AudioFromPath`
|
||
- `MiMoASRLoader` + above audio → `MiMoASRTranscribe` → STRING
|
||
- `VTT_BuildTranscriptJSON` → `VTT_SaveTranscript`
|
||
- 2 × `PreviewAny` to surface outputs over WebSocket
|
||
3. Submit via `POST /prompt`, stream `executing` / `progress` events over WS.
|
||
4. Pull `transcript_json` from `history.outputs.preview_json.text` and write to `<--out>/<stem>.json`.
|
||
|
||
The `save` node also persists a timestamped copy to the server's `ComfyUI/output/`.
|
||
|
||
## Troubleshooting
|
||
|
||
**`Connection refused`** — ComfyUI not running, or wrong `--ip`. `curl http://<ip>:8188/`.
|
||
|
||
**`Node error: ... MiMo ASR repo path is empty`** — Either set `MIMO_ASR_REPO` on the server, or pass `--repo-path` with an absolute path.
|
||
|
||
**`Node error: ... model path does not exist`** — Check `--model-path` and `--tokenizer-path` are absolute paths that exist on the **server** (not your local machine).
|
||
|
||
**`No transcript output`** — script dumps the first 2KB of `/history` to stderr. Most common: MiMoASR not installed, or `flash-attn` install silently failed.
|
||
|
||
**First run takes 30–60s with no progress output** — that's MiMo loading its 8B model into VRAM. Subsequent runs in the same ComfyUI session hit the cache.
|