Federico Volovnik-JoseandClaude Opus 4.7
0435efa858
A full fp16 cast on the audio_tokenizer saved ~3 GB VRAM but produced silent audio: fp16 lacks the dynamic range LayerNorm/RMSNorm need to keep their normalization statistics meaningful, and the decoder collapsed to near-zero output. Now: cast all weights to the resolved dtype, then explicitly cast LayerNorm/GroupNorm/RMSNorm back to fp32. A forward pre-hook on every leaf-ish module aligns incoming float tensors with each module's own param dtype, so: - fp32 wav input -> fp16 Linear: hook casts to fp16 - fp16 hidden state -> fp32 LayerNorm: hook casts to fp32 - fp32 norm output -> fp16 next layer: that layer's hook casts to fp16 Tested with a tiny mixed Linear/Conv1d/LayerNorm stand-in: fp32 inputs flow through fp16 Linear, fp16 hidden states reach fp32 norm cleanly. End-to-end audio quality has to be verified on the real model. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ComfyUI-MOSS-TTS
ComfyUI custom nodes for MOSS-TTS — a 1.7B parameter zero-shot text-to-speech and voice-cloning model.
Installation
- Clone this repo into
ComfyUI/custom_nodes/:cd ComfyUI/custom_nodes git clone https://github.com/Eutectico/ComfyUI-MOSS-TTS.git - Install Python dependencies (ComfyUI does this automatically on next start, or you can run it manually):
pip install -r ComfyUI-MOSS-TTS/requirements.txt - Restart ComfyUI. The nodes appear under the
MOSS-TTS/category in the node menu.
The first generation downloads ~13 GB of model weights from HuggingFace. Subsequent runs use the cached weights.
Transformers version
MOSS-TTS' remote code (loaded via trust_remote_code=True) is calibrated for transformers >= 4.50, < 4.58. Earlier releases miss newer APIs the model expects (MODALITY_TO_BASE_CLASS_MAPPING, PreTrainedConfig); later releases (especially the 5.x line) have removed APIs the model still calls (_get_initial_cache_position, etc.). The package contains compatibility shims that paper over a few specific gaps, but staying inside the supported range produces the cleanest results.
If the shipped audio sounds like noise rather than speech, the most common cause is that an unsupported transformers version is installed. Confirm with:
pip show transformers
and pin to a 4.5x version if needed.
Language support
MOSS-TTS was trained primarily on Chinese. Without a voice reference, the default voice tends to render any input with Chinese-leaning phonetics — including English text. For English (or any non-Chinese language) output, supply a clean 5–15 second reference clip in the target language via the MOSS-TTS Voice Reference node.
Nodes
| Node | Category | Purpose |
|---|---|---|
MOSS-TTS Loader |
MOSS-TTS/loaders |
Loads the model + processor, holds them in a module-level cache. |
MOSS-TTS Voice Reference |
MOSS-TTS/voice |
Wraps an AUDIO input or filesystem path into a voice-reference container. |
MOSS-TTS Generate |
MOSS-TTS/generation |
Generates speech from text (with optional voice reference). |
MOSS-TTS Unload |
MOSS-TTS/utility |
Frees the model from VRAM. Optional; place at the end of a workflow if needed. |
Quality Presets
The Generate node has a preset dropdown:
| Preset | RVQ Layers | Notes |
|---|---|---|
Custom |
from widgets | Uses the individual sampling sliders as-is. |
Fast (8 RVQ) |
8 | Longest audio on T4 (~12 min); lower quality. |
Balanced (16 RVQ) |
16 | Default. ~8 min on T4. |
High Quality (24 RVQ) |
24 | ~5 min on T4. |
Maximum (32 RVQ) |
32 | Best quality, ~4 min on T4. |
When preset is anything other than Custom, the preset's values override the individual sampling widgets at execution time.
Hardware
Defaults target T4-class GPUs (~16 GB VRAM, FP16, sdpa attention). If you have ≥24 GB VRAM, you may want to set dtype=bf16 and increase max_tokens.
Smoke test
- Drop in a
MOSS-TTS Loadernode (defaults are fine). - Drop in a
MOSS-TTS Generatenode, connectmodeland type any text. - Connect
audiotoSave Audio(built-in). - Run. First run downloads weights (~5–8 min); subsequent runs are fast.
Voice cloning
Connect a Load Audio node → MOSS-TTS Voice Reference → MOSS-TTS Generate.voice. A 5–15 second clean reference clip works well.
Credits
- Model: MOSS-TTS by OpenMOSS Team
- Source notebook: AIQUEST (
MOSS_TTS_Voice_Cloning.ipynb) - ComfyUI integration: this repo
License
MIT — see LICENSE.