ForgeAI HeartMuLa — ComfyUI Custom Nodes

All-in-one music generation node for ComfyUI, powered by HeartMuLa.

Works on 16 GB VRAM GPUs (RTX 4060 Ti, RTX 5070 Ti, etc.) with built-in 4-bit quantization.

Nodes

ForgeAI Music Generator

Generate music from lyrics and genre tags in a single node.

• Inputs: Lyrics, Tags (genre/style), Duration, Quantization, Temperature, Top-K, CFG Scale, Output Format
• Outputs: Audio (ComfyUI AUDIO type) + File Path (saved WAV/MP3)
• Quantization: none / 4-bit NF4 / 8-bit (select in node)

ForgeAI Lyrics Transcriber

Transcribe lyrics from audio using Whisper.

• Input: Audio (ComfyUI AUDIO type)
• Output: Transcribed lyrics text

Tag Guide

HeartMuLa uses comma-separated tags to control the style of generated music. Genre is the most important tag — always put it first.

Tag Format

genre:pop, emotional, synth, warm, female voice

Available Tag Categories

Category	Examples	Priority
Genre	pop, rock, electronic, jazz, reggae, j-pop	Highest
Timbre	synth, acoustic, warm, bright, deep	High
Gender	male voice, female voice	High
Mood	emotional, energetic, chill, dark, happy	Medium
Instrument	piano, guitar, bass, drums, strings	Medium
Scene	dance, ballad, anthem, lullaby	Low
Region	latin, french, japanese, korean	Low
Tempo	fast tempo, slow tempo, moderate tempo	Low

Structure Tags (in Lyrics field)

Use structure tags in your lyrics to control song sections:

[intro]
[verse]
Your verse lyrics here...
[chorus]
Your chorus lyrics here...
[bridge]
Bridge lyrics...
[outro]

Multi-Language Lyrics

HeartMuLa supports lyrics in multiple languages including English, Spanish, French, Japanese, and more. Simply write your lyrics in the desired language.

CFG Scale Guide

CFG (Classifier-Free Guidance) controls how strictly the model follows your tags. Range: 1.0 — 10.0

CFG	Best For	Notes
2.0	Pop, Ballads, Emotional	Sweet spot for clean vocals
3.0	Rock, Latin, Uptempo	More energy, stronger instrumentation
4.0+	Electronic, Dance, EDM	Bold sound, may introduce artifacts
5.0+	Experimental	Use with caution

Recommendation: Start with CFG 2.0 and increase if you need more energy or harder genres.

Installation

1. Clone into ComfyUI custom_nodes

cd ComfyUI/custom_nodes
git clone https://github.com/PavonicAI/ForgeAI-HeartMuLa.git

2. Install dependencies

pip install -r ForgeAI-HeartMuLa/requirements.txt

3. Download models

Recommended: Use our pre-quantized 4-bit checkpoint (fits on 16 GB GPUs):

Download ForgeAI/HeartMuLa-3B-4bit and place in ComfyUI/models/HeartMuLa/HeartMuLa-oss-3B/

You also need HeartCodec + tokenizer from the original repo:

ComfyUI/models/HeartMuLa/
  ├── HeartMuLa-oss-3B/    ← our 4-bit checkpoint OR original weights
  ├── HeartCodec-oss/       ← from original repo
  ├── tokenizer.json        ← from original repo
  └── gen_config.json       ← from original repo

Compatibility Fixes Included

All fixes are applied automatically — no manual patching needed:

Issue	Status
ignore_mismatched_sizes (transformers 5.x)	Fixed
RoPE cache not built (torchtune >= 0.5)	Fixed
OOM on 16 GB GPUs	Fixed (model CPU offload)
torchcodec missing (torchaudio >= 2.10)	Fixed (uses soundfile)
CFG 1.0 crash (batch_size mismatch)	Fixed (auto-clamp to 1.1)

Hardware Tested

• NVIDIA RTX 5070 Ti (16 GB) with 4-bit quantization
• Stable for hours of continuous generation
• ~13 GB VRAM during generation, ~8 GB during encoding

License

Apache-2.0

Credits

• Original HeartMuLa model by HeartMuLa Team
• ComfyUI nodes & compatibility fixes by ForgeAI