aadebuger-ComfyUI-VideoToText/cli/README.md

video_to_text_cli.py

Submit a video to a remote ComfyUI server running ComfyUI-VideoToText + ComfyUI-MiMoASR and dump the JSON transcript locally.

Install

uv pip install --system websocket-client requests

Basic usage

# Upload a local file and transcribe with Auto language
uv run python video_to_text_cli.py /path/to/lecture.mp4 \
    --ip 10.8.0.12 \
    --model-path /big_disk/models/MiMo-V2.5-ASR \
    --tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer

# Pure Chinese content
uv run python video_to_text_cli.py speech_zh.mp4 \
    --ip 10.8.0.12 \
    --model-path /big_disk/models/MiMo-V2.5-ASR \
    --tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
    --language Chinese

# Reference a file already in ComfyUI/input (no re-upload)
uv run python video_to_text_cli.py already_uploaded.mp4 \
    --ip 10.8.0.12 \
    --model-path /big_disk/models/MiMo-V2.5-ASR \
    --tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer

# Dump workflow without running — feed into UI for debugging
uv run python video_to_text_cli.py video.mp4 \
    --model-path /big_disk/models/MiMo-V2.5-ASR \
    --tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
    --dump-workflow /tmp/wf.json

Skip retyping paths

Add an alias to your shell:

# ~/.bashrc
alias v2t='uv run python /path/to/ComfyUI-VideoToText/cli/video_to_text_cli.py \
    --ip 10.8.0.12 \
    --model-path /big_disk/models/MiMo-V2.5-ASR \
    --tokenizer-path /big_disk/models/MiMo-Audio-Tokenizer \
    --repo-path /opt/repos/MiMo-V2.5-ASR'

# Then:
v2t my_video.mp4 --language Auto
v2t lecture.mp4 --language Chinese --out ./transcripts

All flags

Flag	Default	Purpose
video (positional)	—	Local path OR filename already in ComfyUI/input.
--ip	127.0.0.1	ComfyUI host.
--port	8188	ComfyUI port.
--server	—	Full host:port (overrides --ip / --port).
--model-path	required	Absolute path to MiMo-V2.5-ASR on the server.
--tokenizer-path	required	Absolute path to MiMo-Audio-Tokenizer on the server.
--repo-path	""	Absolute path to MiMo source repo. Empty if MIMO_ASR_REPO is set on the server.
--force-reload	off	Drop the MiMo loader cache and re-instantiate.
--language	Auto	Auto / Chinese / English.
--sample-rate	16000	ffmpeg output rate before MiMo.
--save-prefix	transcript	Filename prefix on the server side.
--out	./out	Local directory to save the JSON.
--dump-workflow	—	Path to dump generated workflow JSON.
--timeout	1800	WebSocket wait timeout (sec).

How it works

1. Upload the local video to ComfyUI/input/ via POST /upload/image.
2. Build the 9-node API workflow:
   • VTT_LoadVideo → VTT_ExtractAudio → VTT_AudioFromPath
   • MiMoASRLoader + above audio → MiMoASRTranscribe → STRING
   • VTT_BuildTranscriptJSON → VTT_SaveTranscript
   • 2 × PreviewAny to surface outputs over WebSocket
3. Submit via POST /prompt, stream executing / progress events over WS.
4. Pull transcript_json from history.outputs.preview_json.text and write to <--out>/<stem>.json.

The save node also persists a timestamped copy to the server's ComfyUI/output/.

Troubleshooting

Connection refused — ComfyUI not running, or wrong --ip. curl http://<ip>:8188/.

Node error: ... MiMo ASR repo path is empty — Either set MIMO_ASR_REPO on the server, or pass --repo-path with an absolute path.

Node error: ... model path does not exist — Check --model-path and --tokenizer-path are absolute paths that exist on the server (not your local machine).

No transcript output — script dumps the first 2KB of /history to stderr. Most common: MiMoASR not installed, or flash-attn install silently failed.

First run takes 30–60s with no progress output — that's MiMo loading its 8B model into VRAM. Subsequent runs in the same ComfyUI session hit the cache.