From 324b15952b5d5b137d56f92c6f366dd1d947ec7e Mon Sep 17 00:00:00 2001 From: Gong Junmin <1836678486@qq.com> Date: Sat, 28 Feb 2026 21:35:25 +0800 Subject: [PATCH 1/2] feat: refactor nodes into modular workflow with sample_mode and auto audio2code - Replace monolithic AceStepMusicGen node with modular architecture: AceStepText2MusicGenParams, AceStepSettings, AceStepText2MusicServer, AceStepAudioCodes, AceStepShowText - Merge sample_mode into GenParams node with JS-driven widget visibility (sample_mode ON: sample_query/is_instrumental/language only; sample_mode OFF: full manual text2music controls) - Move conditionally-hidden fields to optional inputs with defaults to prevent ComfyUI validation errors when JS hides widgets - Remove standalone audio2code and example_generate server nodes; audio_codes now returned as a third output of Text2MusicServer - Add frontend JS for dynamic widget hide/show, API key masking, server mode toggling (cloud/local), and show-text display - Include pre-built text2music.json workflow - Rewrite README with full node reference and usage guide Made-with: Cursor --- README.md | 160 +++++-- __init__.py | 20 +- js/ace_button.js | 269 ++++++++--- nodes.py | 971 ++++++++++++++++++++++++++------------ workflows/text2music.json | 225 +++++++++ 5 files changed, 1229 insertions(+), 416 deletions(-) create mode 100644 workflows/text2music.json diff --git a/README.md b/README.md index d7fa857..04eaeea 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,12 @@ # ACE-Step-ComfyUI -Official ComfyUI node for [ACE-Step](https://acemusic.ai) AI music generation via ACE-Step API. - -## Features - -- **Text to Music** — Generate music from text descriptions and lyrics -- **Cover** — Create cover versions from source audio with style control -- **Repaint** — Regenerate specific sections of an audio track - +ComfyUI nodes for [ACE-Step](https://acemusic.ai) AI music generation — text-to-music, cover/remix, repaint, and LLM-powered sample generation. ## Installation ### Via ComfyUI Manager -Search for `ACE-Step-ComfyUI` in ComfyUI Manager and install. +Search for **ACE-Step-ComfyUI** in ComfyUI Manager and install. ### Manual Installation @@ -28,46 +21,129 @@ Restart ComfyUI after installation. ## Setup -1. Get your API key from [acemusic.ai](https://acemusic.ai/api-key) -2. Set the key using **one** of these methods: - - Enter it in the `api_key` widget on the node (auto-saved locally after first input, displayed as `●●●●` for security). The key is **not** stored in workflow JSON, so sharing workflows will not leak your API key - - Set the environment variable `ACESTEP_API_KEY` +### Cloud Mode (default) -> **Note:** The API key is saved in the `.config/` folder under the node directory. Do not share this folder with others. +1. Get your API key from [acemusic.ai/api-key](https://acemusic.ai/api-key) +2. In any Server node, set **mode** → `cloud` and paste your key into **api_key** +3. Or set the environment variable `ACESTEP_API_KEY` -## Node: ACE-Step Music Generate +### Local Mode -Located in: **api node > audio > AceStep** +1. Start the ACE-Step OpenRouter server locally: + ```bash + cd ACE-Step-1.5 + uv run acestep-openrouter --host 0.0.0.0 --port 8002 + ``` +2. In any Server node, set **mode** → `local` +3. The URL auto-fills to `http://127.0.0.1:8002`; **api_key** and **Get API Key** button are hidden in local mode -### Inputs +## Workflows -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| server_url | STRING | `https://api.acemusic.ai` | ACE-Step API server URL | -| prompt | STRING | | Music description / caption | -| lyrics | STRING | | Song lyrics (leave empty for instrumental) | -| task_type | ENUM | `text2music` | `text2music`, `cover`, or `repaint` | -| instrumental | BOOLEAN | `false` | Generate instrumental only | -| duration | FLOAT | `30.0` | Audio duration in seconds (1–300) | -| bpm | INT | `0` | Beats per minute (0 = auto) | -| key_scale | STRING | | Musical key and scale (e.g. `C major`) | -| time_signature | STRING | | Time signature (e.g. `4/4`, `3/4`) | -| vocal_language | ENUM | `en` | Vocal language | -| sample_mode | BOOLEAN | `false` | Use LLM to generate prompt/lyrics from query | -| use_format | BOOLEAN | `false` | Use LLM to enhance caption and lyrics | -| seed | STRING | `-1` | Random seed (-1 = random, comma-separated for batch) | -| temperature | FLOAT | `0.85` | Sampling temperature | -| guidance_scale | FLOAT | `7.0` | Guidance scale | -| src_audio | AUDIO | | Source audio for cover/repaint tasks | -| ref_audio | AUDIO | | Reference audio for style/timbre guidance | -| api_key | STRING | | API key (auto-saved locally) | +Two ready-to-use workflow files are provided. Drag any `.json` file into ComfyUI to load it. -### Outputs +| File | Use Case | +|------|----------| +| `workflows/text2music.json` | Text-to-music, cover, remix, and repaint | +| `workflows/example_generate.json` | LLM auto-generates caption/lyrics/metadata, then synthesizes | -| Output | Type | Description | -|--------|------|-------------| -| audio | AUDIO | Generated audio waveform | -| info | STRING | Text response from the API | +### Workflow 1: Text2music / Cover / Remix / Repaint + +**`workflows/text2music.json`** + +``` +Load Audio (refer) ─── refer_audio ──┐ +Load Audio (src) ──┬── src_audio ────┤ + │ ▼ + audio2code Server Text2music Gen Params ── gen_params ──► Text2music Server ──► Save Audio + │ ▲ ▲ │ + Audio Codes ── audio_codes┘ Settings ─┘ Show Text +``` + +**How to use:** + +- **Text-to-music (basic):** Fill in **caption** and **lyrics** in Gen Params, leave everything else default. Click Queue Prompt. +- **With reference audio:** Load an audio file in "Load Audio (refer audio)", connect it to Gen Params → `refer_audio`. This guides the style/timbre. +- **Cover mode:** Load the source song in "Load Audio (src audio)", connect it to either: + - Gen Params → `src_audio` (sends raw audio directly), OR + - audio2code Server → Audio Codes → Gen Params → `audio_codes` (converts to codes first) + - Task type automatically switches to `cover` when `src_audio` or `audio_codes` is connected + - **cover_strength** — noise injection strength (0 = clean cover, higher = more variation) + - **remix_strength** — how much of the original to preserve (1.0 = full, 0 = none) +- **Repaint mode:** Connect source audio to Gen Params → `src_audio`, then check **is_repaint** in Gen Params. Set **repaint_start** and **repaint_end** (in seconds) to define the region to regenerate. Fill in **caption** and **lyrics** for the regenerated region. +- Cover/repaint mode automatically disables thinking, CoT caption, and CoT language — no LM is used. + +**Settings tips:** + +- **auto** toggle (ON by default): lets the LM decide bpm, key, duration, time signature. Turn OFF to set them manually. +- **thinking** = true: enables 5Hz LM audio code generation (higher quality, slower) +- **seed**: set a number for reproducible results, `-1` for random + +### Workflow 2: Example Generate (LLM Auto-Generation) + +**`workflows/example_generate.json`** + +``` +Example Generate Server ── gen_params ──► Text2music Server ──► Save Audio + │ ▲ │ + └── info ──► Show Text (LLM params) Settings ─┘ Show Text (server info) +``` + +**How to use:** + +1. Type a natural language description in **sample_query** (e.g. "upbeat pop song about summer") +2. Set **language** and **is_instrumental** +3. Click Queue Prompt — the LLM generates caption, lyrics, bpm, key, duration, etc., then synthesizes music +4. Check "LLM Generated Params" to see what the LLM produced + +This is the equivalent of Gradio's "Simple mode" — fully automatic. + +## Node Reference + +### Server Nodes + +These nodes call the ACE-Step API. Each has **mode** (cloud/local), **server_url**, and **api_key** fields. + +| Node | Description | Outputs | +|------|-------------|---------| +| **audio2code Server** | Converts source audio → audio code tokens | `audio_codes`, `info` | +| **Example Generate Server** | LLM generates caption/lyrics/metadata from a query | `gen_params`, `caption`, `lyrics`, `info` | +| **Text2music Server** | Generates music (text2music/cover/remix/repaint) from gen_params + settings | `audio`, `info` | + +### Parameter Nodes + +| Node | Description | Key Fields | +|------|-------------|------------| +| **Text2music Gen Params** | Parameters for text2music / cover / remix / repaint | `caption`, `lyrics`, `vocal_language`, `bpm`, `key`, `duration`, `time_signature`, `auto`, `cover_strength`, `remix_strength`, `is_repaint`, `repaint_start`, `repaint_end` + optional `refer_audio`, `src_audio`, `audio_codes` | +| **Settings** | LM + DiT inference hyperparameters | `seed`, `thinking`, `use_cot_caption`, `use_cot_language`, `temperature`, `lm_cfg_scale`, `lm_top_p`, `lm_top_k`, `dit_guidance_scale`, `dit_inference_steps`, `dit_infer_method` | + +### Utility Nodes + +| Node | Description | +|------|-------------| +| **Audio Codes** | Editable passthrough for audio codes — paste manually or receive from audio2code Server. Displays received codes in the text area after execution. | +| **Show Text** | Displays any STRING input as multiline text with auto-sizing. | + +### Built-in ComfyUI Nodes Used + +- **LoadAudio** — load audio files from disk +- **SaveAudio** — save generated audio to disk + +## Task Mode Summary + +| Mode | Trigger | LM Used? | Key Parameters | +|------|---------|----------|----------------| +| **text2music** | No src_audio, no audio_codes, is_repaint OFF | Yes (if thinking=true) | caption, lyrics, bpm, key, duration | +| **cover** | src_audio connected OR audio_codes non-empty | No (auto-disabled) | cover_strength, remix_strength | +| **repaint** | src_audio connected AND is_repaint checked | No (auto-disabled) | repaint_start, repaint_end | +| **example generate** | Use Example Generate workflow | Yes (LLM generates params) | sample_query, language, is_instrumental | + +## Tips + +- **Cover/Repaint auto-safety:** When task is `cover` or `repaint`, `thinking`, `use_cot_caption`, `use_cot_language`, and `use_cot_metas` are all forced to `false` regardless of Settings values. The LM is not used for these tasks. +- **Auto metas:** When `auto` is ON in Gen Params, the `use_cot_metas` flag is sent as `true`, letting the LM infer bpm/key/duration/time_signature. When `auto` is OFF, your manual values are sent directly. +- **is_repaint toggle:** When OFF (default), `repaint_start` and `repaint_end` are hidden. When ON, they appear. Repaint requires `src_audio` to be connected. +- Workflows can be combined — users can copy nodes between workflows to create custom pipelines. +- Use **Show Text** nodes connected to `info` outputs to inspect generation parameters and server responses. ## License diff --git a/__init__.py b/__init__.py index 1889fc0..35bdbbc 100644 --- a/__init__.py +++ b/__init__.py @@ -1,11 +1,25 @@ -from .nodes import AceStepMusicGen +from .nodes import ( + AceStepAudioCodes, + AceStepText2MusicGenParams, + AceStepSettings, + AceStepText2MusicServer, + AceStepShowText, +) NODE_CLASS_MAPPINGS = { - "AceStepMusicGen": AceStepMusicGen, + "AceStepAudioCodes": AceStepAudioCodes, + "AceStepText2MusicGenParams": AceStepText2MusicGenParams, + "AceStepSettings": AceStepSettings, + "AceStepText2MusicServer": AceStepText2MusicServer, + "AceStepShowText": AceStepShowText, } NODE_DISPLAY_NAME_MAPPINGS = { - "AceStepMusicGen": "ACE-Step Music Generate", + "AceStepAudioCodes": "ACE-Step Audio Codes", + "AceStepText2MusicGenParams": "ACE-Step Text2music Gen Params", + "AceStepSettings": "ACE-Step Settings", + "AceStepText2MusicServer": "ACE-Step Text2music Server", + "AceStepShowText": "ACE-Step Show Text", } WEB_DIRECTORY = "./js" diff --git a/js/ace_button.js b/js/ace_button.js index 9606f9b..6a9a997 100644 --- a/js/ace_button.js +++ b/js/ace_button.js @@ -1,5 +1,9 @@ import { app } from "../../scripts/app.js"; +const SERVER_NODES = [ + "AceStepText2MusicServer", +]; + function hideWidget(node, widget) { if (widget._hidden) return; widget._hidden = true; @@ -7,6 +11,11 @@ function hideWidget(node, widget) { widget._origComputeSize = widget.computeSize; widget.type = "hidden"; widget.computeSize = () => [0, -4]; + const idx = node.widgets.indexOf(widget); + if (idx >= 0) { + widget._origIndex = idx; + node.widgets.splice(idx, 1); + } } function showWidget(node, widget) { @@ -16,71 +25,213 @@ function showWidget(node, widget) { widget.computeSize = widget._origComputeSize; delete widget._origType; delete widget._origComputeSize; + if (!node.widgets.includes(widget)) { + const idx = Math.min(widget._origIndex ?? node.widgets.length, node.widgets.length); + node.widgets.splice(idx, 0, widget); + } + delete widget._origIndex; +} + +function recalcSize(node, extraH = 0) { + const sz = node.computeSize(); + node.setSize([Math.max(sz[0], node.size[0]), sz[1] + extraH]); +} + +function setupApiKeyMasking(node) { + const w = node.widgets.find(w => w.name === "api_key"); + if (!w) return; + let realKey = ""; + w.serializeValue = async () => realKey; + w.callback = function (value) { + if (!value) { realKey = ""; return; } + if (/^●+$/.test(value)) return; + realKey = value; + w.value = "●".repeat(Math.min(value.length, 16)); + }; +} + +function setupServerNode(node) { + setupApiKeyMasking(node); + + const modeW = node.widgets.find(w => w.name === "mode"); + const urlW = node.widgets.find(w => w.name === "server_url"); + const keyW = node.widgets.find(w => w.name === "api_key"); + if (!modeW || !urlW) return; + + const btnW = node.addWidget("button", "🔑 Get API Key", null, () => { + window.open("https://acemusic.ai/api-key", "_blank"); + }); + btnW.serialize = false; + + const CLOUD = "https://api.acemusic.ai"; + const LOCAL = "http://127.0.0.1:8002"; + + function update() { + const isLocal = modeW.value === "local"; + if (urlW.value === CLOUD || urlW.value === LOCAL || !urlW.value.trim()) { + urlW.value = isLocal ? LOCAL : CLOUD; + } + if (keyW) { isLocal ? hideWidget(node, keyW) : showWidget(node, keyW); } + isLocal ? hideWidget(node, btnW) : showWidget(node, btnW); + recalcSize(node); + } + + const orig = modeW.callback; + modeW.callback = function (...args) { + if (orig) orig.apply(this, args); + update(); + }; + update(); +} + +function setupGenParamsToggles(node) { + // Master list: canonical widget order, captured once. + const masterWidgets = node.widgets.slice(); + const origTypes = new Map(); + const origCompute = new Map(); + for (const w of masterWidgets) { + origTypes.set(w, w.type); + origCompute.set(w, w.computeSize); + } + + const byName = (n) => masterWidgets.find(w => w.name === n); + const sampleModeW = byName("sample_mode"); + const autoW = byName("auto"); + const repaintW = byName("is_repaint"); + + const TEXT_ROWS = { caption: 3, lyrics: 7, sample_query: 4 }; + let extraHeight = 0; + for (const w of masterWidgets) { + if (TEXT_ROWS[w.name] && w.inputEl) { + const defaultRows = w.inputEl.rows || 3; + w.inputEl.rows = TEXT_ROWS[w.name]; + w.inputEl.style.minHeight = (TEXT_ROWS[w.name] * 1.4) + "em"; + extraHeight += (TEXT_ROWS[w.name] - defaultRows) * 18; + } + } + + function computeVisible() { + const vis = new Set(); + const isSample = sampleModeW && sampleModeW.value; + + vis.add("sample_mode"); + + if (isSample) { + vis.add("sample_query"); + vis.add("is_instrumental"); + vis.add("vocal_language"); + } else { + vis.add("caption"); + vis.add("lyrics"); + vis.add("vocal_language"); + vis.add("auto"); + vis.add("is_repaint"); + + const isAuto = autoW && autoW.value; + if (!isAuto) { + vis.add("bpm"); + vis.add("key"); + vis.add("duration"); + vis.add("time_signature"); + } + + const isRepaint = repaintW && repaintW.value; + if (isRepaint) { + vis.add("repaint_start"); + vis.add("repaint_end"); + } else { + vis.add("cover_strength"); + vis.add("remix_strength"); + } + } + return vis; + } + + function update() { + const vis = computeVisible(); + + // Rebuild node.widgets in master order, only including visible ones. + node.widgets.length = 0; + for (const w of masterWidgets) { + if (vis.has(w.name)) { + w.type = origTypes.get(w); + w.computeSize = origCompute.get(w); + w._hidden = false; + node.widgets.push(w); + } else { + w.type = "hidden"; + w.computeSize = () => [0, -4]; + w._hidden = true; + } + } + + recalcSize(node, extraHeight); + } + + for (const toggleW of [sampleModeW, autoW, repaintW]) { + if (!toggleW) continue; + const orig = toggleW.callback; + toggleW.callback = function (...args) { + if (orig) orig.apply(this, args); + update(); + }; + } + update(); +} + +function setupShowText(node) { + const el = document.createElement("textarea"); + el.readOnly = true; + el.style.cssText = + "width:100%;background:transparent;color:inherit;border:none;" + + "resize:none;font:inherit;padding:4px;opacity:0.85;outline:none;" + + "white-space:pre-wrap;word-wrap:break-word;overflow-y:auto;"; + el.rows = 4; + + const textWidget = node.addDOMWidget("ace_text_output", "customtext", el, { + serialize: false, + getValue() { return el.value; }, + setValue(v) { el.value = v || ""; }, + }); + + node.onExecuted = function (data) { + if (data?.text?.[0] != null) { + el.value = data.text[0]; + const lines = (data.text[0].match(/\n/g) || []).length + 1; + const rows = Math.max(4, Math.min(lines, 20)); + el.rows = rows; + const h = Math.max(120, Math.min(rows * 18 + 80, 500)); + node.setSize([Math.max(node.size[0], 300), h]); + } + }; +} + +function setupAudioCodes(node) { + node.onExecuted = function (data) { + const codes = data?.audio_codes?.[0]; + if (codes == null) return; + const w = node.widgets.find(w => w.name === "audio_codes"); + if (w) { + w.value = codes; + recalcSize(node); + } + }; } app.registerExtension({ name: "AceStep.NodeExtensions", - nodeCreated(node) { - if (node.comfyClass !== "AceStepMusicGen") return; - - const REPAINT_WIDGETS = ["repainting_start", "repainting_end"]; - const COVER_WIDGETS = ["audio_cover_strength"]; - - function updateVisibility() { - const taskWidget = node.widgets.find(w => w.name === "task_type"); - if (!taskWidget) return; - - const task = taskWidget.value; - - for (const w of node.widgets) { - if (REPAINT_WIDGETS.includes(w.name)) { - task === "repaint" ? showWidget(node, w) : hideWidget(node, w); - } else if (COVER_WIDGETS.includes(w.name)) { - task === "cover" ? showWidget(node, w) : hideWidget(node, w); - } - } - - const sz = node.computeSize(); - node.setSize([Math.max(sz[0], node.size[0]), sz[1]]); + if (SERVER_NODES.includes(node.comfyClass)) { + requestAnimationFrame(() => setupServerNode(node)); } - - requestAnimationFrame(() => { - const taskWidget = node.widgets.find(w => w.name === "task_type"); - if (taskWidget) { - const origCallback = taskWidget.callback; - taskWidget.callback = function (...args) { - if (origCallback) origCallback.apply(this, args); - updateVisibility(); - }; - } - updateVisibility(); - }); - - // --- API Key masking --- - const apiKeyWidget = node.widgets.find(w => w.name === "api_key"); - if (apiKeyWidget) { - let realKey = ""; - - // Return real key for execution, not the masked display value - apiKeyWidget.serializeValue = async () => realKey; - - apiKeyWidget.callback = function (value) { - if (!value) { - realKey = ""; - return; - } - // Ignore if user submitted the mask unchanged - if (/^●+$/.test(value)) return; - realKey = value; - apiKeyWidget.value = "●".repeat(Math.min(value.length, 16)); - }; + if (node.comfyClass === "AceStepText2MusicGenParams") { + requestAnimationFrame(() => setupGenParamsToggles(node)); + } + if (node.comfyClass === "AceStepShowText") { + requestAnimationFrame(() => setupShowText(node)); + } + if (node.comfyClass === "AceStepAudioCodes") { + requestAnimationFrame(() => setupAudioCodes(node)); } - - // --- Get API Key button --- - const btn = node.addWidget("button", "🔑 Get API Key", null, () => { - window.open("https://acemusic.ai/api-key", "_blank"); - }); - btn.serialize = false; }, }); diff --git a/nodes.py b/nodes.py index b6d9a47..f9d22f2 100644 --- a/nodes.py +++ b/nodes.py @@ -10,6 +10,10 @@ import torch import requests +# ============================================================================= +# Persistent API key storage +# ============================================================================= + _CONFIG_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), ".config") _API_KEY_FILE = os.path.join(_CONFIG_DIR, "api_key") @@ -28,358 +32,701 @@ def _load_api_key(): return "" -class AceStepMusicGen: - """Generate music via ACE-Step OpenRouter-compatible API server.""" +# ============================================================================= +# HTTP / audio helpers +# ============================================================================= - CATEGORY = "api node/audio/AceStep" - FUNCTION = "generate" - RETURN_TYPES = ("AUDIO", "STRING") - RETURN_NAMES = ("audio", "info") +def _resolve_api_key(api_key: str) -> str: + resolved = api_key.strip() if api_key else "" + if resolved: + _save_api_key(resolved) + os.environ["ACESTEP_API_KEY"] = resolved + else: + resolved = os.environ.get("ACESTEP_API_KEY", "") or _load_api_key() + return resolved + + +def _make_headers(api_key: str) -> dict: + headers = { + "Content-Type": "application/json", + "User-Agent": "AceMusic-ComfyUI/2.0", + } + resolved = _resolve_api_key(api_key) + if resolved: + headers["Authorization"] = f"Bearer {resolved}" + return headers + + +def _post_json(url: str, body: dict, headers: dict, timeout: int = 600) -> dict: + try: + resp = requests.post( + url, + data=json.dumps(body, ensure_ascii=False).encode("utf-8"), + headers=headers, + timeout=timeout, + ) + resp.raise_for_status() + return resp.json() + except requests.exceptions.HTTPError: + raise RuntimeError(f"API error {resp.status_code}: {resp.text}") + except requests.exceptions.ConnectionError as e: + raise RuntimeError(f"Cannot connect to server: {e}") + except requests.exceptions.Timeout: + raise RuntimeError("Request timed out") + + +def _tensor_to_wav_b64(waveform, sample_rate) -> str: + n_channels = waveform.shape[0] + clamped = waveform.clamp(-1.0, 1.0) + pcm = (clamped * 32767).to(torch.int16) + interleaved = pcm.T.contiguous().reshape(-1) + raw = struct.pack(f"<{interleaved.numel()}h", *interleaved.tolist()) + buf = io.BytesIO() + with wave.open(buf, "wb") as wf: + wf.setnchannels(n_channels) + wf.setsampwidth(2) + wf.setframerate(sample_rate) + wf.writeframes(raw) + return base64.b64encode(buf.getvalue()).decode("utf-8") + + +def _encode_audio_b64(audio) -> str: + waveform = audio["waveform"].squeeze(0) + return _tensor_to_wav_b64(waveform, audio["sample_rate"]) + + +def _decode_audio_data_url(data_url: str): + if "," in data_url: + b64_data = data_url.split(",", 1)[1] + else: + b64_data = data_url + audio_bytes = base64.b64decode(b64_data) + buf = io.BytesIO(audio_bytes) + with wave.open(buf, "rb") as wf: + n_channels = wf.getnchannels() + sampwidth = wf.getsampwidth() + sample_rate = wf.getframerate() + n_frames = wf.getnframes() + raw = wf.readframes(n_frames) + if sampwidth == 2: + samples = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0 + elif sampwidth == 4: + samples = np.frombuffer(raw, dtype=np.int32).astype(np.float32) / 2147483648.0 + else: + raise RuntimeError(f"Unsupported WAV sample width: {sampwidth}") + samples = samples.reshape(n_frames, n_channels).T + tensor = torch.from_numpy(samples).unsqueeze(0) + return tensor, sample_rate + + +def _parse_audio_response(result: dict): + choices = result.get("choices", []) + audio_items = [] + text_content = "" + if choices: + message = choices[0].get("message", {}) + text_content = message.get("content", "") + audio_items = message.get("audio", []) + else: + audio_items = result.get("audio", []) + text_content = json.dumps(result.get("metadata", {}), ensure_ascii=False) + if result.get("lyrics"): + text_content += f"\n\nLyrics:\n{result['lyrics']}" + if not audio_items: + raise RuntimeError( + f"API returned no audio. Response: {text_content or json.dumps(result)}" + ) + waveforms = [] + sample_rate = None + for item in audio_items: + url = "" + if isinstance(item, dict): + url = item.get("audio_url", {}).get("url", "") or item.get("url", "") + if not url: + continue + wf, sr = _decode_audio_data_url(url) + waveforms.append(wf) + sample_rate = sr + if not waveforms: + raise RuntimeError("API returned no valid audio data") + max_len = max(w.shape[-1] for w in waveforms) + padded = [] + for wf in waveforms: + if wf.shape[-1] < max_len: + wf = torch.nn.functional.pad(wf, (0, max_len - wf.shape[-1])) + padded.append(wf) + audio_tensor = torch.cat(padded, dim=0) + return {"waveform": audio_tensor, "sample_rate": sample_rate}, text_content + + +def _build_multimodal_content(prompt: str, audio_list: list) -> list | str: + if not audio_list: + return prompt + parts = [] + if prompt: + parts.append({"type": "text", "text": prompt}) + for audio in audio_list: + b64 = _encode_audio_b64(audio) + parts.append({ + "type": "input_audio", + "input_audio": {"data": b64, "format": "wav"}, + }) + return parts + + +# ============================================================================= +# Constants +# ============================================================================= + +VALID_LANGUAGES = [ + "en", "zh", "ja", "ko", "es", "fr", "de", "pt", "ru", "it", + "ar", "az", "bg", "bn", "ca", "cs", "da", "el", "fa", "fi", + "he", "hi", "hr", "ht", "hu", "id", "is", "la", "lt", "ms", + "ne", "nl", "no", "pa", "pl", "ro", "sa", "sk", "sr", "sv", + "sw", "ta", "te", "th", "tl", "tr", "uk", "ur", "vi", "yue", + "unknown", +] + +DEFAULT_CAPTION = ( + "tight and groovy disco-funk track driven by a hyper-articulate slap " + "bassline weaving syncopated sixteenth-note grooves with percussive thumb " + "pops and ghost-note textures, locked into a crisp four-on-the-floor drum " + "machine beat. Clean, funky guitar provides sparse chord stabs to leave " + "sonic space for the bass, while filtered synth pads swell subtly in the " + "background. The smooth male lead vocal glides over the infectious groove. " + "The arrangement builds through verses and catchy choruses, then strips " + "down to drums and bass for an extended, harmonically adventurous solo " + "section: the bassist unleashes a technically explosive showcase\u2014rapid " + "double-thumb slaps morph into tapped harmonics, chromatic walking lines " + "resolve into chordal double-stops, and envelope-filter sweeps cascade " + "into distorted octave leaps. After the solo\u2019s climax, wah-drenched funk " + "guitar re-enters for a call-and-response exchange with the bass before " + "the full band drops into a final chorus and a gradual fade out, with the " + "bassline\u2019s final harmonic ringing into silence." +) + +DEFAULT_LYRICS = ( + "[Intro]\n" + "[Heavy guitar riff and drums]\n" + "[Vocal scream] Yeah!\n" + "\n" + "[Guitar Solo]\n" + "\n" + "[Verse 1]\n" + "\n" + "[Pre-Chorus]\n" + "\n" + "[Chorus]\n" + "\n" + "[Verse 2]\n" + "\n" + "[Pre-Chorus]\n" + "\n" + "[Chorus]\n" + "\n" + "[Bridge]\n" + "\n" + "[Guitar Solo]\n" + "\n" + "[Chorus]\n" + "\n" + "[Outro]\n" + "[Song ends abruptly]" +) + + +# ============================================================================= +# Node: Audio Codes (editable passthrough) +# ============================================================================= + +class AceStepAudioCodes: + """Editable audio codes passthrough. + Paste codes manually, or receive from Text2Music Server generation output.""" + + CATEGORY = "ACE-Step" + FUNCTION = "process" + RETURN_TYPES = ("STRING",) + RETURN_NAMES = ("audio_codes",) + OUTPUT_NODE = True @classmethod def INPUT_TYPES(cls): return { "required": { - "server_url": ("STRING", { - "default": "https://api.acemusic.ai", - "tooltip": "ACE-Step API server base URL", - }), - "prompt": ("STRING", { + "audio_codes": ("STRING", { "default": "", "multiline": True, - "tooltip": "Music description / caption", + "tooltip": "Audio codes (<|audio_code_N|> tokens). " + "Edit manually or receive from audio_codes_in.", }), }, "optional": { - "lyrics": ("STRING", { + "audio_codes_in": ("STRING", { "default": "", - "multiline": True, - "tooltip": "Song lyrics (leave empty for instrumental)", - }), - "task_type": (["text2music", "cover", "repaint"], { - "default": "text2music", - }), - "instrumental": ("BOOLEAN", { - "default": False, - "tooltip": "Generate instrumental only (no vocals)", - }), - "duration": ("FLOAT", { - "default": 30.0, - "min": 10.0, - "max": 240.0, - "step": 1.0, - "tooltip": "Audio duration in seconds", - }), - "bpm": ("INT", { - "default": 30, - "min": 30, - "max": 300, - "step": 1, - "tooltip": "Beats per minute (0 = auto)", - }), - "key_scale": ("STRING", { - "default": "", - "tooltip": "Musical key and scale (e.g. 'C major', 'A minor')", - }), - "time_signature": (["2", "3", "4", "6"], { - "default": "4", - "tooltip": "Time signature", - }), - "vocal_language": (["en", "zh", "ja", "ko", "es", "fr", "de", "unknown"], { - "default": "en", + "forceInput": True, + "tooltip": "Auto-fill from Text2Music Server output or paste manually", }), + }, + } + + def process(self, audio_codes="", audio_codes_in=""): + resolved = audio_codes_in.strip() if audio_codes_in and audio_codes_in.strip() else audio_codes.strip() + return {"ui": {"audio_codes": [resolved]}, "result": (resolved,)} + + +# ============================================================================= +# Node: Text2music Gen Params +# ============================================================================= + +class AceStepText2MusicGenParams: + """Generation parameters for text2music / cover / remix / repaint.""" + + CATEGORY = "ACE-Step" + FUNCTION = "build" + RETURN_TYPES = ("ACESTEP_GEN_PARAMS",) + RETURN_NAMES = ("gen_params",) + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { "sample_mode": ("BOOLEAN", { "default": False, - "tooltip": "Use LLM to generate prompt/lyrics from query", + "tooltip": "ON: use sample_query to let LLM generate caption/lyrics/metadata. " + "OFF: use manual caption and lyrics below.", }), - "use_format": ("BOOLEAN", { + "vocal_language": (VALID_LANGUAGES, {"default": "en"}), + }, + "optional": { + "sample_query": ("STRING", { + "default": "", + "multiline": True, + "placeholder": "Describe the music you want (e.g. upbeat pop song about summer)", + "tooltip": "Natural language description for LLM to generate caption/lyrics/metadata", + }), + "is_instrumental": ("BOOLEAN", { "default": False, - "tooltip": "Use LLM to enhance caption and lyrics", + "tooltip": "Generate instrumental only (no vocals).", }), - "seed": ("STRING", { - "default": "-1", - "tooltip": "Random seed (-1 = random)", + "caption": ("STRING", { + "default": DEFAULT_CAPTION, + "multiline": True, + "placeholder": "Music style description (e.g. deep house, dreamy, atmospheric)", + "tooltip": "Music style description / caption", }), - "temperature": ("FLOAT", { - "default": 0.85, - "min": 0.0, - "max": 2.0, - "step": 0.05, + "lyrics": ("STRING", { + "default": DEFAULT_LYRICS, + "multiline": True, + "placeholder": "Song lyrics with [verse], [chorus] tags... (empty = instrumental)", + "tooltip": "Song lyrics (leave empty for instrumental)", }), - "guidance_scale": ("FLOAT", { - "default": 7.0, - "min": 0.0, - "max": 20.0, - "step": 0.5, + "auto": ("BOOLEAN", { + "default": True, + "tooltip": "ON: bpm/key left for LM to decide. " + "OFF: use manual values.", }), - "src_audio": ("AUDIO", { - "tooltip": "Source audio for cover/repaint tasks", + "cover_strength": ("FLOAT", { + "default": 0.0, "min": 0.0, "max": 1.0, "step": 0.05, + "tooltip": "Cover strength (only used when src_audio is connected)", }), - "ref_audio": ("AUDIO", { - "tooltip": "Reference audio for style/timbre guidance", + "remix_strength": ("FLOAT", { + "default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05, + "tooltip": "Remix noise strength (only used when src_audio is connected)", }), - "repainting_start": ("FLOAT", { - "default": 0.0, - "min": 0.0, - "max": 300.0, - "step": 0.1, + "is_repaint": ("BOOLEAN", { + "default": False, + "tooltip": "Enable repaint mode (requires src_audio). " + "Regenerates a time range within the source audio.", + }), + "bpm": ("INT", { + "default": 120, "min": 0, "max": 300, "step": 1, + "tooltip": "Beats per minute (0 = auto)", + }), + "key": ("STRING", { + "default": "", + "tooltip": "e.g. 'C major', 'D minor'", + }), + "duration": ("FLOAT", { + "default": 30.0, "min": -1.0, "max": 600.0, "step": 1.0, + "tooltip": "Duration in seconds (-1 = model decides)", + }), + "time_signature": ("STRING", { + "default": "4", + "tooltip": "Time signature (e.g. 2, 3, 4, 6). Empty = auto.", + }), + "repaint_start": ("FLOAT", { + "default": 0.0, "min": 0.0, "max": 600.0, "step": 0.1, "tooltip": "Repaint region start time in seconds", }), - "repainting_end": ("FLOAT", { - "default": 0.0, - "min": 0.0, - "max": 300.0, - "step": 0.1, + "repaint_end": ("FLOAT", { + "default": 0.0, "min": 0.0, "max": 600.0, "step": 0.1, "tooltip": "Repaint region end time in seconds (0 = full length)", }), - "audio_cover_strength": ("FLOAT", { - "default": 1.0, - "min": 0.0, - "max": 1.0, - "step": 0.05, - "tooltip": "Cover strength (1.0 = full cover)", + "refer_audio": ("AUDIO", { + "tooltip": "Reference audio for style/timbre guidance", }), - "api_key": ("STRING", { + "src_audio": ("AUDIO", { + "tooltip": "Source audio for cover/remix/repaint", + }), + "audio_codes": ("STRING", { "default": "", - "tooltip": "API key (auto-saved locally after first input, also reads ACESTEP_API_KEY env var)", + "forceInput": True, + "tooltip": "Audio codes from Audio Codes node", }), }, } + def build(self, sample_mode, vocal_language, + sample_query="", is_instrumental=False, + caption=DEFAULT_CAPTION, lyrics=DEFAULT_LYRICS, + auto=True, cover_strength=0.0, remix_strength=1.0, is_repaint=False, + bpm=120, key="", duration=30.0, time_signature="4", + repaint_start=0.0, repaint_end=0.0, + refer_audio=None, src_audio=None, audio_codes=""): + codes = audio_codes.strip() if audio_codes else "" + has_src = src_audio is not None + has_codes = bool(codes) + + if is_repaint and has_src: + task_type = "repaint" + elif has_src or has_codes: + task_type = "cover" + else: + task_type = "text2music" + + if sample_mode: + instrumental = is_instrumental + else: + instrumental = not lyrics.strip() + + return ({ + "task_type": task_type, + "sample_mode": sample_mode, + "sample_query": sample_query.strip() if sample_mode else "", + "caption": caption, + "lyrics": lyrics, + "vocal_language": vocal_language, + "instrumental": instrumental, + "bpm": 0 if auto else bpm, + "key_scale": "" if auto else key, + "duration": -1.0 if auto else duration, + "time_signature": "" if auto else time_signature, + "auto_metas": auto, + "cover_strength": cover_strength, + "remix_strength": remix_strength, + "repaint_start": repaint_start, + "repaint_end": repaint_end, + "refer_audio": refer_audio, + "src_audio": src_audio, + "audio_codes": codes, + },) + + +# ============================================================================= +# Node: Settings +# ============================================================================= + +class AceStepSettings: + """Inference settings: LM + DiT parameters.""" + + CATEGORY = "ACE-Step" + FUNCTION = "build" + RETURN_TYPES = ("ACESTEP_SETTINGS",) + RETURN_NAMES = ("settings",) + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "seed": ("STRING", {"default": "-1", "tooltip": "-1 = random"}), + "thinking": ("BOOLEAN", { + "default": True, + "tooltip": "Enable 5Hz LM audio code generation (llm_dit mode)", + }), + "use_cot_caption": ("BOOLEAN", { + "default": True, + "tooltip": "LLM rewrites/enhances caption via CoT", + }), + "use_cot_language": ("BOOLEAN", { + "default": True, + "tooltip": "LLM auto-detects vocal language", + }), + "temperature": ("FLOAT", { + "default": 0.85, "min": 0.0, "max": 2.0, "step": 0.05, + "tooltip": "LM sampling temperature", + }), + "lm_cfg_scale": ("FLOAT", { + "default": 2.0, "min": 1.0, "max": 5.0, "step": 0.1, + "tooltip": "LM classifier-free guidance scale", + }), + "lm_top_p": ("FLOAT", { + "default": 0.9, "min": 0.0, "max": 1.0, "step": 0.01, + "tooltip": "LM nucleus sampling top-p", + }), + "lm_top_k": ("INT", { + "default": 0, "min": 0, "max": 200, "step": 1, + "tooltip": "LM top-k sampling (0 = disabled)", + }), + "dit_guidance_scale": ("FLOAT", { + "default": 7.0, "min": 0.0, "max": 20.0, "step": 0.5, + "tooltip": "DiT classifier-free guidance scale", + }), + "dit_inference_steps": ("INT", { + "default": 8, "min": 1, "max": 200, "step": 1, + "tooltip": "DiT diffusion steps (turbo default = 8)", + }), + "dit_infer_method": (["ode", "sde"], { + "default": "ode", + "tooltip": "DiT ODE/SDE inference method", + }), + }, + } + + def build(self, seed, thinking, use_cot_caption, use_cot_language, + temperature, lm_cfg_scale, lm_top_p, lm_top_k, + dit_guidance_scale, dit_inference_steps, dit_infer_method): + return ({ + "seed": seed, + "thinking": thinking, + "use_cot_caption": use_cot_caption, + "use_cot_language": use_cot_language, + "temperature": temperature, + "lm_cfg_scale": lm_cfg_scale, + "lm_top_p": lm_top_p, + "lm_top_k": lm_top_k, + "dit_guidance_scale": dit_guidance_scale, + "dit_inference_steps": dit_inference_steps, + "dit_infer_method": dit_infer_method, + },) + + +# ============================================================================= +# Shared: build request body for text2music / repaint Server nodes +# ============================================================================= + +def _build_request_body(gen_params: dict, settings: dict) -> dict: + gp = gen_params + st = settings + + is_sample_mode = gp.get("sample_mode", False) + sample_query = gp.get("sample_query", "") + + audio_list = [] + if gp.get("refer_audio") is not None: + audio_list.append(gp["refer_audio"]) + if gp.get("src_audio") is not None: + audio_list.append(gp["src_audio"]) + + prompt = sample_query if is_sample_mode else gp.get("caption", "") + content = _build_multimodal_content(prompt, audio_list) + + task_type = gp.get("task_type", "text2music") + has_src = gp.get("src_audio") is not None + has_codes = bool((gp.get("audio_codes", "") or "").strip()) + if task_type == "text2music" and (has_src or has_codes): + task_type = "cover" + + body = { + "model": "acemusic/acestep-v15-turbo", + "messages": [{"role": "user", "content": content}], + "modalities": ["audio"], + "stream": False, + "task_type": task_type, + "thinking": st.get("thinking", True), + "temperature": st.get("temperature", 0.85), + "top_p": st.get("lm_top_p", 0.9), + "use_cot_caption": st.get("use_cot_caption", True), + "use_cot_language": st.get("use_cot_language", True), + "use_cot_metas": gp.get("auto_metas", True), + "guidance_scale": st.get("dit_guidance_scale", 7.0), + "audio_config": { + "format": "wav", + "vocal_language": gp.get("vocal_language", "en"), + "instrumental": gp.get("instrumental", False), + }, + } + + if is_sample_mode: + body["sample_mode"] = True + + lm_cfg = st.get("lm_cfg_scale") + if lm_cfg is not None: + body["lm_cfg_scale"] = lm_cfg + lm_top_k = st.get("lm_top_k", 0) + if lm_top_k and lm_top_k > 0: + body["top_k"] = lm_top_k + + steps = st.get("dit_inference_steps") + if steps is not None: + body["inference_steps"] = steps + method = st.get("dit_infer_method") + if method: + body["infer_method"] = method + + duration = gp.get("duration", -1.0) + if duration and duration > 0: + body["audio_config"]["duration"] = duration + + bpm = gp.get("bpm", 0) + if bpm and bpm > 0: + body["audio_config"]["bpm"] = bpm + + key_scale = (gp.get("key_scale", "") or "").strip() + if key_scale: + body["audio_config"]["key_scale"] = key_scale + + time_sig = gp.get("time_signature", "") + if time_sig: + body["audio_config"]["time_signature"] = time_sig + + lyrics = gp.get("lyrics", "") + if lyrics and not is_sample_mode: + body["lyrics"] = lyrics + + seed_str = str(st.get("seed", "-1")).strip().split(",")[0].strip() + if seed_str and seed_str != "-1": + try: + int(seed_str) + except ValueError: + raise RuntimeError(f"Invalid seed: '{seed_str}'") + body["seed"] = seed_str + + # Cover/remix: server handles audio2code internally + if task_type == "cover": + body["thinking"] = False + body["use_cot_caption"] = False + body["use_cot_language"] = False + body["use_cot_metas"] = False + rs = gp.get("remix_strength", 1.0) + if rs is not None: + body["audio_cover_strength"] = rs + cs = gp.get("cover_strength", 0.0) + if cs is not None and cs > 0: + body["cover_noise_strength"] = cs + + # Repaint: LM is not used + if task_type == "repaint": + body["thinking"] = False + body["use_cot_caption"] = False + body["use_cot_language"] = False + body["use_cot_metas"] = False + body["repainting_start"] = gp.get("repaint_start", 0.0) + rend = gp.get("repaint_end", 0.0) + if rend and rend > 0: + body["repainting_end"] = rend + + audio_codes = gp.get("audio_codes", "") + if body.get("thinking", False): + pass + elif audio_codes: + body["audio_codes"] = audio_codes + + return body + + +# ============================================================================= +# Node: Text2music Server +# ============================================================================= + +class AceStepText2MusicServer: + """Text2music / cover / remix / repaint server. + Inputs: serve_config fields + gen_params + settings. + Outputs: audio + info + audio_codes.""" + + CATEGORY = "ACE-Step" + FUNCTION = "generate" + RETURN_TYPES = ("AUDIO", "STRING", "STRING") + RETURN_NAMES = ("audio", "info", "audio_codes") + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mode": (["cloud", "local"], {"default": "cloud"}), + "server_url": ("STRING", {"default": "https://api.acemusic.ai"}), + "gen_params": ("ACESTEP_GEN_PARAMS", {}), + "settings": ("ACESTEP_SETTINGS", {}), + }, + "optional": { + "api_key": ("STRING", {"default": ""}), + }, + } + @classmethod def IS_CHANGED(cls, **kwargs): return float("NaN") - def generate( - self, - server_url, - prompt, - lyrics="", - task_type="text2music", - instrumental=False, - duration=30.0, - bpm=0, - key_scale="", - time_signature="4", - vocal_language="en", - sample_mode=False, - use_format=False, - batch_size=1, - seed="-1", - temperature=0.85, - guidance_scale=7.0, - src_audio=None, - ref_audio=None, - repainting_start=0.0, - repainting_end=0.0, - audio_cover_strength=1.0, - api_key="", - ): - url = server_url.rstrip("/") + "/v1/chat/completions" + def generate(self, mode, server_url, gen_params, settings, api_key=""): + url = server_url.strip() + if not url: + url = "http://127.0.0.1:8002" if mode == "local" else "https://api.acemusic.ai" + base = url.rstrip("/") + headers = _make_headers(api_key) - # Build message content with audio inputs - # API routing: cover/repaint → audio[0]=src, audio[1]=ref; text2music → audio[0]=ref - audio_list = [] - if task_type in ("cover", "repaint"): - if src_audio is not None: - audio_list.append(src_audio) - if ref_audio is not None: - audio_list.append(ref_audio) - else: - if ref_audio is not None: - audio_list.append(ref_audio) + body = _build_request_body(gen_params, settings) + actual_task = body.get("task_type", "text2music") + result = _post_json(f"{base}/v1/chat/completions", body, headers) + audio, text_content = _parse_audio_response(result) - content = self._build_content(prompt, audio_list) + info_parts = [] + gp = gen_params + param_summary = ( + f"task: {actual_task}\n" + f"caption: {gp.get('caption', '')}\n" + f"lyrics: {gp.get('lyrics', '')[:80]}{'...' if len(gp.get('lyrics', '')) > 80 else ''}\n" + f"bpm: {gp.get('bpm', 0)} key: {gp.get('key_scale', '')} " + f"duration: {gp.get('duration', -1.0)} time_sig: {gp.get('time_signature', '')}\n" + f"vocal_language: {gp.get('vocal_language', '')} " + f"instrumental: {gp.get('instrumental', False)}" + ) + if actual_task == "cover": + param_summary += ( + f"\ncover_strength: {gp.get('cover_strength', 0.0)} " + f"remix_strength: {gp.get('remix_strength', 1.0)}" + ) + if gp.get("audio_codes"): + param_summary += f"\naudio_codes: {len(gp['audio_codes'])} chars" + if actual_task == "repaint": + param_summary += ( + f"\nrepaint_start: {gp.get('repaint_start', 0.0)} " + f"repaint_end: {gp.get('repaint_end', 0.0)}" + ) + info_parts.append(f"=== Gen Params ===\n{param_summary}") - # Build request body - body = { - "model": "acemusic/acestep-v15-turbo", - "messages": [{"role": "user", "content": content}], - "modalities": ["audio"], - "stream": False, - "temperature": temperature, - "guidance_scale": guidance_scale, - "batch_size": batch_size, - "task_type": task_type, - "sample_mode": sample_mode, - "use_format": use_format, - "audio_config": { - "duration": duration, - "format": "wav", - "vocal_language": vocal_language, - "instrumental": instrumental, + if text_content: + info_parts.append(f"\n=== Server Response ===\n{text_content}") + out_codes = "" + choices = result.get("choices", []) + if choices: + msg = choices[0].get("message", {}) + out_codes = msg.get("audio_codes", "") or "" + if out_codes: + info_parts.append(f"\n=== Audio Codes ({len(out_codes)} chars) ===") + info = "\n".join(info_parts) + + return (audio, info, out_codes) + + +# ============================================================================= +# Node: Show Text (minimal text display, no external dependencies) +# ============================================================================= + +class AceStepShowText: + """Display any STRING input as text.""" + + CATEGORY = "ACE-Step" + FUNCTION = "show" + RETURN_TYPES = ("STRING",) + RETURN_NAMES = ("text",) + OUTPUT_NODE = True + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "text": ("STRING", {"forceInput": True}), }, } - # Seed: "-1" means random, otherwise validate and take the first value - seed_str = seed.strip().split(",")[0].strip() - if seed_str and seed_str != "-1": - try: - int(seed_str) - except ValueError: - raise RuntimeError(f"Invalid seed value: '{seed_str}', must be an integer") - body["seed"] = seed_str - if bpm > 0: - body["audio_config"]["bpm"] = bpm - body["audio_config"]["key_scale"] = key_scale.strip() or None - body["audio_config"]["time_signature"] = time_signature - if lyrics: - body["lyrics"] = lyrics - - # Repaint/cover specific parameters (only when relevant) - if task_type == "repaint": - body["repainting_start"] = repainting_start - if repainting_end > 0.0: - body["repainting_end"] = repainting_end - if task_type == "cover": - body["audio_cover_strength"] = audio_cover_strength - - # HTTP request – resolve API key: widget input > env var > saved config - resolved_key = api_key.strip() if api_key else "" - if resolved_key: - _save_api_key(resolved_key) - os.environ["ACESTEP_API_KEY"] = resolved_key - else: - resolved_key = os.environ.get("ACESTEP_API_KEY", "") or _load_api_key() - headers = { - "Content-Type": "application/json", - "User-Agent": "AceMusic-ComfyUI/1.0", - } - if resolved_key: - headers["Authorization"] = f"Bearer {resolved_key}" - - try: - resp = requests.post( - url, - data=json.dumps(body, ensure_ascii=False).encode("utf-8"), - headers=headers, - timeout=600, - ) - resp.raise_for_status() - result = resp.json() - except requests.exceptions.HTTPError as e: - raise RuntimeError(f"API error {resp.status_code}: {resp.text}") - except requests.exceptions.ConnectionError as e: - raise RuntimeError(f"Cannot connect to {server_url}: {e}") - except requests.exceptions.Timeout: - raise RuntimeError(f"Request to {server_url} timed out") - - # Parse response - choices = result.get("choices", []) - if not choices: - raise RuntimeError("API returned no choices") - - message = choices[0].get("message", {}) - text_content = message.get("content", "") - audio_items = message.get("audio", []) - - if not audio_items: - raise RuntimeError(f"API returned no audio. Response: {text_content}") - - # Decode all audio items and batch them - waveforms = [] - sample_rate = None - for item in audio_items: - audio_url = item.get("audio_url", {}).get("url", "") - if not audio_url: - continue - wf, sr = self._decode_audio_data_url(audio_url) - # wf shape: (1, channels, samples) - waveforms.append(wf) - sample_rate = sr - - if not waveforms: - raise RuntimeError("API returned no valid audio data") - - # Pad to same length and concat along batch dim - max_len = max(w.shape[-1] for w in waveforms) - padded = [] - for wf in waveforms: - if wf.shape[-1] < max_len: - pad_size = max_len - wf.shape[-1] - wf = torch.nn.functional.pad(wf, (0, pad_size)) - padded.append(wf) - - # (N, channels, samples) - audio_tensor = torch.cat(padded, dim=0) - - return ( - {"waveform": audio_tensor, "sample_rate": sample_rate}, - text_content, - ) - - def _build_content(self, prompt, audio_list): - """Build message content, optionally with multimodal audio inputs.""" - if not audio_list: - return prompt - - # Multimodal: text + input_audio(s) - parts = [] - - if prompt: - parts.append({"type": "text", "text": prompt}) - - for audio in audio_list: - waveform = audio["waveform"].squeeze(0) # (channels, samples) - sr = audio["sample_rate"] - b64 = self._tensor_to_wav_b64(waveform, sr) - parts.append({ - "type": "input_audio", - "input_audio": {"data": b64, "format": "wav"}, - }) - - return parts - - def _decode_audio_data_url(self, data_url): - """Decode base64 data URL to torch audio tensor using stdlib wave.""" - if "," in data_url: - b64_data = data_url.split(",", 1)[1] - else: - b64_data = data_url - - audio_bytes = base64.b64decode(b64_data) - buf = io.BytesIO(audio_bytes) - - with wave.open(buf, "rb") as wf: - n_channels = wf.getnchannels() - sampwidth = wf.getsampwidth() - sample_rate = wf.getframerate() - n_frames = wf.getnframes() - raw = wf.readframes(n_frames) - - # Use numpy for fast PCM decoding - if sampwidth == 2: - samples = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0 - elif sampwidth == 4: - samples = np.frombuffer(raw, dtype=np.int32).astype(np.float32) / 2147483648.0 - else: - raise RuntimeError(f"Unsupported WAV sample width: {sampwidth}") - - # Interleaved -> (n_frames, n_channels) -> (n_channels, n_frames) - samples = samples.reshape(n_frames, n_channels).T - # Add batch dim: (1, n_channels, n_frames) - tensor = torch.from_numpy(samples).unsqueeze(0) - return tensor, sample_rate - - def _tensor_to_wav_b64(self, waveform, sample_rate): - """Encode torch tensor to WAV base64 string using stdlib wave.""" - # waveform shape: (channels, samples) - n_channels = waveform.shape[0] - n_frames = waveform.shape[1] - - # Clamp and convert to 16-bit PCM - clamped = waveform.clamp(-1.0, 1.0) - pcm = (clamped * 32767).to(torch.int16) - - # Interleave channels: (channels, samples) -> (samples * channels,) - interleaved = pcm.T.contiguous().reshape(-1) - raw = struct.pack(f"<{interleaved.numel()}h", *interleaved.tolist()) - - buf = io.BytesIO() - with wave.open(buf, "wb") as wf: - wf.setnchannels(n_channels) - wf.setsampwidth(2) - wf.setframerate(sample_rate) - wf.writeframes(raw) - - return base64.b64encode(buf.getvalue()).decode("utf-8") + def show(self, text): + return {"ui": {"text": [text]}, "result": (text,)} diff --git a/workflows/text2music.json b/workflows/text2music.json new file mode 100644 index 0000000..38379e9 --- /dev/null +++ b/workflows/text2music.json @@ -0,0 +1,225 @@ +{ + "id": "ffe0e07f-9772-45cb-845d-bf5666d27930", + "revision": 1, + "last_node_id": 12, + "last_link_id": 10, + "nodes": [ + { + "id": 5, + "type": "AceStepText2MusicGenParams", + "pos": [40, 40], + "size": [420, 500], + "flags": {}, + "order": 0, + "mode": 0, + "inputs": [ + { "name": "refer_audio", "shape": 7, "type": "AUDIO", "link": null }, + { "name": "src_audio", "shape": 7, "type": "AUDIO", "link": null }, + { "name": "audio_codes", "shape": 7, "type": "STRING", "link": null } + ], + "outputs": [ + { + "name": "gen_params", + "type": "ACESTEP_GEN_PARAMS", + "slot_index": 0, + "links": [1] + } + ], + "title": "Text2music Gen Params", + "properties": { "Node name for S&R": "AceStepText2MusicGenParams" }, + "widgets_values": [ + false, + "en" + ] + }, + { + "id": 6, + "type": "AceStepSettings", + "pos": [40, 590], + "size": [420, 298], + "flags": {}, + "order": 1, + "mode": 0, + "inputs": [], + "outputs": [ + { + "name": "settings", + "type": "ACESTEP_SETTINGS", + "slot_index": 0, + "links": [2] + } + ], + "title": "Settings", + "properties": { "Node name for S&R": "AceStepSettings" }, + "widgets_values": [ + "-1", true, true, true, 0.85, 2, 0.9, 0, 7, 8, "ode" + ] + }, + { + "id": 11, + "type": "LoadAudio", + "pos": [500, 40], + "size": [270, 136], + "flags": {}, + "order": 2, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "AUDIO", "type": "AUDIO", "slot_index": 0, "links": null } + ], + "title": "Load Audio (refer audio)", + "properties": { "Node name for S&R": "LoadAudio" }, + "widgets_values": ["", null, null] + }, + { + "id": 2, + "type": "LoadAudio", + "pos": [500, 220], + "size": [270, 136], + "flags": {}, + "order": 3, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "AUDIO", "type": "AUDIO", "slot_index": 0, "links": null } + ], + "title": "Load Audio (src audio)", + "properties": { "Node name for S&R": "LoadAudio" }, + "widgets_values": ["", null, null] + }, + { + "id": 7, + "type": "AceStepText2MusicServer", + "pos": [500, 420], + "size": [340, 130], + "flags": {}, + "order": 4, + "mode": 0, + "inputs": [ + { "name": "gen_params", "type": "ACESTEP_GEN_PARAMS", "link": 1 }, + { "name": "settings", "type": "ACESTEP_SETTINGS", "link": 2 } + ], + "outputs": [ + { "name": "audio", "type": "AUDIO", "slot_index": 0, "links": [3] }, + { "name": "info", "type": "STRING", "slot_index": 1, "links": [4] }, + { "name": "audio_codes", "type": "STRING", "slot_index": 2, "links": [5] } + ], + "title": "Text2music Server", + "properties": { "Node name for S&R": "AceStepText2MusicServer" }, + "widgets_values": ["cloud", "https://api.acemusic.ai"] + }, + { + "id": 8, + "type": "SaveAudio", + "pos": [500, 600], + "size": [340, 112], + "flags": {}, + "order": 5, + "mode": 0, + "inputs": [ + { "name": "audio", "type": "AUDIO", "link": 3 } + ], + "outputs": [], + "title": "Save Audio", + "properties": { "Node name for S&R": "SaveAudio" }, + "widgets_values": ["ace-step/text2music"] + }, + { + "id": 9, + "type": "AceStepShowText", + "pos": [500, 760], + "size": [340, 160], + "flags": {}, + "order": 6, + "mode": 0, + "inputs": [ + { "name": "text", "type": "STRING", "link": 4 } + ], + "outputs": [ + { "name": "text", "type": "STRING", "links": null } + ], + "title": "Generation Info", + "properties": { "Node name for S&R": "AceStepShowText" }, + "widgets_values": [""] + }, + { + "id": 4, + "type": "AceStepAudioCodes", + "pos": [880, 40], + "size": [300, 200], + "flags": {}, + "order": 7, + "mode": 0, + "inputs": [ + { "name": "audio_codes_in", "shape": 7, "type": "STRING", "link": 5 } + ], + "outputs": [ + { "name": "audio_codes", "type": "STRING", "slot_index": 0, "links": null } + ], + "title": "Audio Codes", + "properties": { "Node name for S&R": "AceStepAudioCodes" }, + "widgets_values": [""] + }, + { + "id": 12, + "type": "Note", + "pos": [880, 280], + "size": [300, 340], + "flags": {}, + "order": 8, + "mode": 0, + "inputs": [], + "outputs": [], + "title": "Usage Guide", + "properties": {}, + "widgets_values": [ + "=== Text2music ===\nFill caption + lyrics, click Queue Prompt.\nOr toggle sample_mode ON and describe\nthe music you want in sample_query.\n\n=== Cover / Remix ===\nConnect src_audio to Gen Params.\nServer auto-converts audio to codes.\nAdjust cover/remix_strength.\n\n=== Repaint ===\nConnect src_audio, check is_repaint,\nset repaint_start / repaint_end.\n\n=== Reference Audio ===\nConnect refer_audio for style guidance.\n\n=== Auto Toggle ===\nON = LM decides bpm/key/duration.\nOFF = use your manual values.\n\n=== Audio Codes ===\nCodes from generation are output to\nthe Audio Codes node for reuse." + ], + "color": "#232", + "bgcolor": "#353" + } + ], + "links": [ + [1, 5, 0, 7, 0, "ACESTEP_GEN_PARAMS"], + [2, 6, 0, 7, 1, "ACESTEP_SETTINGS"], + [3, 7, 0, 8, 0, "AUDIO"], + [4, 7, 1, 9, 0, "STRING"], + [5, 7, 2, 4, 0, "STRING"] + ], + "groups": [ + { + "id": 1, + "title": "Parameters", + "bounding": [20, -10, 460, 920], + "color": "#3f789e", + "font_size": 18, + "flags": {} + }, + { + "id": 2, + "title": "Audio Input & Generation", + "bounding": [480, -10, 380, 960], + "color": "#3f789e", + "font_size": 18, + "flags": {} + }, + { + "id": 3, + "title": "Output & Codes", + "bounding": [860, -10, 340, 660], + "color": "#3f789e", + "font_size": 18, + "flags": {} + } + ], + "config": {}, + "extra": { + "ds": { + "scale": 0.8, + "offset": [30, 30] + }, + "workflowRendererVersion": "LG", + "frontendVersion": "1.39.19" + }, + "version": 0.4 +} From 20bd05453ed3b3be304c37f7390c33c2d0aae881 Mon Sep 17 00:00:00 2001 From: Gong Junmin <1836678486@qq.com> Date: Sat, 28 Feb 2026 21:37:53 +0800 Subject: [PATCH 2/2] docs: rewrite README with local setup guide and updated usage, add default sample_query - Rewrite Local Mode section with step-by-step install and server startup instructions - Update Usage Guide to reflect current single-workflow architecture (no more audio2code Server or Example Generate nodes) - Add detailed sections for sample mode, manual mode, cover/remix, repaint, reference audio, and settings - Update Node Reference table to match the 5 current nodes - Remove references to deleted nodes (audio2code Server, Example Generate Server) - Add default sample_query: "a funk rock song with groovy bass and punchy drums" Made-with: Cursor --- README.md | 221 ++++++++++++++++++++++++++++++------------------------ nodes.py | 4 +- 2 files changed, 123 insertions(+), 102 deletions(-) diff --git a/README.md b/README.md index 04eaeea..7879010 100644 --- a/README.md +++ b/README.md @@ -24,126 +24,147 @@ Restart ComfyUI after installation. ### Cloud Mode (default) 1. Get your API key from [acemusic.ai/api-key](https://acemusic.ai/api-key) -2. In any Server node, set **mode** → `cloud` and paste your key into **api_key** +2. In the **Text2music Server** node, set **mode** → `cloud` and paste your key into **api_key** 3. Or set the environment variable `ACESTEP_API_KEY` +> The key is auto-saved locally (in `.config/` under the node directory) and displayed as `●●●●` for security. It is **not** stored in workflow JSON. + ### Local Mode -1. Start the ACE-Step OpenRouter server locally: - ```bash - cd ACE-Step-1.5 - uv run acestep-openrouter --host 0.0.0.0 --port 8002 - ``` -2. In any Server node, set **mode** → `local` -3. The URL auto-fills to `http://127.0.0.1:8002`; **api_key** and **Get API Key** button are hidden in local mode +Local mode lets you run the ACE-Step inference server on your own machine (requires a GPU with sufficient VRAM). -## Workflows +**1. Install ACE-Step 1.5** -Two ready-to-use workflow files are provided. Drag any `.json` file into ComfyUI to load it. - -| File | Use Case | -|------|----------| -| `workflows/text2music.json` | Text-to-music, cover, remix, and repaint | -| `workflows/example_generate.json` | LLM auto-generates caption/lyrics/metadata, then synthesizes | - -### Workflow 1: Text2music / Cover / Remix / Repaint - -**`workflows/text2music.json`** - -``` -Load Audio (refer) ─── refer_audio ──┐ -Load Audio (src) ──┬── src_audio ────┤ - │ ▼ - audio2code Server Text2music Gen Params ── gen_params ──► Text2music Server ──► Save Audio - │ ▲ ▲ │ - Audio Codes ── audio_codes┘ Settings ─┘ Show Text +```bash +git clone https://github.com/ace-step/ACE-Step-1.5.git +cd ACE-Step-1.5 ``` -**How to use:** +Install dependencies (choose one): -- **Text-to-music (basic):** Fill in **caption** and **lyrics** in Gen Params, leave everything else default. Click Queue Prompt. -- **With reference audio:** Load an audio file in "Load Audio (refer audio)", connect it to Gen Params → `refer_audio`. This guides the style/timbre. -- **Cover mode:** Load the source song in "Load Audio (src audio)", connect it to either: - - Gen Params → `src_audio` (sends raw audio directly), OR - - audio2code Server → Audio Codes → Gen Params → `audio_codes` (converts to codes first) - - Task type automatically switches to `cover` when `src_audio` or `audio_codes` is connected - - **cover_strength** — noise injection strength (0 = clean cover, higher = more variation) - - **remix_strength** — how much of the original to preserve (1.0 = full, 0 = none) -- **Repaint mode:** Connect source audio to Gen Params → `src_audio`, then check **is_repaint** in Gen Params. Set **repaint_start** and **repaint_end** (in seconds) to define the region to regenerate. Fill in **caption** and **lyrics** for the regenerated region. -- Cover/repaint mode automatically disables thinking, CoT caption, and CoT language — no LM is used. +```bash +# Using uv (recommended) +uv sync -**Settings tips:** - -- **auto** toggle (ON by default): lets the LM decide bpm, key, duration, time signature. Turn OFF to set them manually. -- **thinking** = true: enables 5Hz LM audio code generation (higher quality, slower) -- **seed**: set a number for reproducible results, `-1` for random - -### Workflow 2: Example Generate (LLM Auto-Generation) - -**`workflows/example_generate.json`** - -``` -Example Generate Server ── gen_params ──► Text2music Server ──► Save Audio - │ ▲ │ - └── info ──► Show Text (LLM params) Settings ─┘ Show Text (server info) +# Or using pip +pip install -e . ``` -**How to use:** +**2. Start the local server** -1. Type a natural language description in **sample_query** (e.g. "upbeat pop song about summer") -2. Set **language** and **is_instrumental** -3. Click Queue Prompt — the LLM generates caption, lyrics, bpm, key, duration, etc., then synthesizes music -4. Check "LLM Generated Params" to see what the LLM produced +```bash +# Using uv +uv run acestep-openrouter --host 0.0.0.0 --port 8002 -This is the equivalent of Gradio's "Simple mode" — fully automatic. +# Or if installed via pip +acestep-openrouter --host 0.0.0.0 --port 8002 +``` + +The server will download model weights on first launch and then listen on `http://127.0.0.1:8002`. + +**3. Configure ComfyUI** + +In the **Text2music Server** node, set **mode** → `local`. The URL auto-fills to `http://127.0.0.1:8002` and the **api_key** field is hidden since no key is needed locally. + +## Usage Guide + +Drag `workflows/text2music.json` into ComfyUI to load the ready-to-use workflow. + +``` + ┌── refer_audio ──┐ +Load Audio (refer) ──────────┘ │ + ▼ + Text2music Gen Params ── gen_params ──► Text2music Server ──► Save Audio + ▲ ▲ │ +Load Audio (src) ──── src_audio ───────────────┘ Settings ┘ Show Text +Audio Codes ────────── audio_codes ────────────┘ +``` + +### Quick Start: Text-to-Music + +1. Open the workflow. **sample_mode** is OFF by default — you are in manual mode. +2. Fill in **caption** (music style description) and **lyrics** (with `[verse]`, `[chorus]` tags). Leave lyrics empty for instrumental. +3. Click **Queue Prompt**. The generated audio appears in **Save Audio** and generation info in **Show Text**. + +### Sample Mode (LLM Auto-Generation) + +Instead of writing caption and lyrics yourself, let the LLM generate everything from a simple description: + +1. Toggle **sample_mode** → ON in the Gen Params node. +2. Fill in **sample_query** (e.g. `"a funk rock song with groovy bass and punchy drums"`). +3. Choose **vocal_language** and set **is_instrumental** if you want instrumental only. +4. Click **Queue Prompt**. The LLM generates caption, lyrics, bpm, key, duration, and time signature automatically, then synthesizes the music. + +> In sample mode, only three fields are shown: `sample_query`, `vocal_language`, and `is_instrumental`. All other parameters are decided by the LLM. + +### Manual Mode Controls + +When **sample_mode** is OFF, you have full manual control: + +| Field | Description | +|-------|-------------| +| **caption** | Music style / genre description | +| **lyrics** | Song lyrics (empty = instrumental) | +| **vocal_language** | Language for vocals | +| **auto** | ON (default): let the LM decide bpm, key, duration, time signature. OFF: set them manually below. | +| **bpm** | Beats per minute (shown when auto=OFF) | +| **key** | Musical key, e.g. `C major` (shown when auto=OFF) | +| **duration** | Duration in seconds (shown when auto=OFF) | +| **time_signature** | e.g. `4`, `3`, `6` (shown when auto=OFF) | +| **is_repaint** | Enable repaint mode (see below) | +| **cover_strength** | Noise injection strength for cover (0 = clean cover) | +| **remix_strength** | How much original audio to preserve (1.0 = full) | + +### Cover / Remix Mode + +1. Load the source song via **Load Audio** and connect it to Gen Params → `src_audio`. +2. Alternatively, connect pre-extracted audio codes via **Audio Codes** → Gen Params → `audio_codes`. +3. The task type automatically switches to `cover` when `src_audio` or `audio_codes` is connected. +4. Adjust **cover_strength** and **remix_strength** to control the output. + +> The server automatically converts source audio to audio codes internally when needed. The generated `audio_codes` are also returned as a third output of Text2music Server. + +### Repaint Mode + +1. Connect source audio to Gen Params → `src_audio`. +2. Toggle **is_repaint** → ON. The **repaint_start** and **repaint_end** fields appear. +3. Set the time range (in seconds) for the region to regenerate. +4. Fill in **caption** and **lyrics** for the regenerated section. + +### Reference Audio + +Connect a reference audio file to Gen Params → `refer_audio` to guide the style and timbre of the generated music. This works with any mode. + +### Settings Node + +The **Settings** node controls inference hyperparameters: + +| Field | Default | Description | +|-------|---------|-------------| +| **seed** | `-1` | Random seed (`-1` = random, set a number for reproducibility) | +| **thinking** | `true` | Enable 5Hz LM audio code generation (higher quality, slower) | +| **use_cot_caption** | `true` | LM chain-of-thought for caption refinement | +| **use_cot_language** | `true` | LM chain-of-thought for language detection | +| **temperature** | `0.85` | Sampling temperature | +| **lm_cfg_scale** | `1.0` | LM classifier-free guidance scale | +| **dit_guidance_scale** | `3.5` | DiT guidance scale | +| **dit_inference_steps** | `60` | Number of DiT denoising steps | ## Node Reference -### Server Nodes - -These nodes call the ACE-Step API. Each has **mode** (cloud/local), **server_url**, and **api_key** fields. - -| Node | Description | Outputs | -|------|-------------|---------| -| **audio2code Server** | Converts source audio → audio code tokens | `audio_codes`, `info` | -| **Example Generate Server** | LLM generates caption/lyrics/metadata from a query | `gen_params`, `caption`, `lyrics`, `info` | -| **Text2music Server** | Generates music (text2music/cover/remix/repaint) from gen_params + settings | `audio`, `info` | - -### Parameter Nodes - -| Node | Description | Key Fields | -|------|-------------|------------| -| **Text2music Gen Params** | Parameters for text2music / cover / remix / repaint | `caption`, `lyrics`, `vocal_language`, `bpm`, `key`, `duration`, `time_signature`, `auto`, `cover_strength`, `remix_strength`, `is_repaint`, `repaint_start`, `repaint_end` + optional `refer_audio`, `src_audio`, `audio_codes` | -| **Settings** | LM + DiT inference hyperparameters | `seed`, `thinking`, `use_cot_caption`, `use_cot_language`, `temperature`, `lm_cfg_scale`, `lm_top_p`, `lm_top_k`, `dit_guidance_scale`, `dit_inference_steps`, `dit_infer_method` | - -### Utility Nodes - -| Node | Description | -|------|-------------| -| **Audio Codes** | Editable passthrough for audio codes — paste manually or receive from audio2code Server. Displays received codes in the text area after execution. | -| **Show Text** | Displays any STRING input as multiline text with auto-sizing. | - -### Built-in ComfyUI Nodes Used - -- **LoadAudio** — load audio files from disk -- **SaveAudio** — save generated audio to disk - -## Task Mode Summary - -| Mode | Trigger | LM Used? | Key Parameters | -|------|---------|----------|----------------| -| **text2music** | No src_audio, no audio_codes, is_repaint OFF | Yes (if thinking=true) | caption, lyrics, bpm, key, duration | -| **cover** | src_audio connected OR audio_codes non-empty | No (auto-disabled) | cover_strength, remix_strength | -| **repaint** | src_audio connected AND is_repaint checked | No (auto-disabled) | repaint_start, repaint_end | -| **example generate** | Use Example Generate workflow | Yes (LLM generates params) | sample_query, language, is_instrumental | +| Node | Description | Inputs | Outputs | +|------|-------------|--------|---------| +| **Text2music Gen Params** | Build generation parameters. Supports sample_mode (LLM auto) and manual mode. | `sample_mode`, `vocal_language`, + optional fields | `gen_params` | +| **Settings** | Inference hyperparameters for LM + DiT. | `seed`, `thinking`, `temperature`, etc. | `settings` | +| **Text2music Server** | Calls the ACE-Step API to generate music. | `gen_params`, `settings`, `server_url`, `api_key`, `mode` | `audio`, `info`, `audio_codes` | +| **Audio Codes** | Editable passthrough for audio codes. Paste manually or receive from Text2music Server. | `audio_codes_in` (optional) | `audio_codes` | +| **Show Text** | Displays any STRING input as a read-only text area. | `text` | `text` (passthrough) | ## Tips -- **Cover/Repaint auto-safety:** When task is `cover` or `repaint`, `thinking`, `use_cot_caption`, `use_cot_language`, and `use_cot_metas` are all forced to `false` regardless of Settings values. The LM is not used for these tasks. -- **Auto metas:** When `auto` is ON in Gen Params, the `use_cot_metas` flag is sent as `true`, letting the LM infer bpm/key/duration/time_signature. When `auto` is OFF, your manual values are sent directly. -- **is_repaint toggle:** When OFF (default), `repaint_start` and `repaint_end` are hidden. When ON, they appear. Repaint requires `src_audio` to be connected. -- Workflows can be combined — users can copy nodes between workflows to create custom pipelines. -- Use **Show Text** nodes connected to `info` outputs to inspect generation parameters and server responses. +- **Cover/Repaint auto-safety:** When the task is `cover` or `repaint`, `thinking`, `use_cot_caption`, and `use_cot_language` are forced to `false` regardless of Settings values — the LM is not used for these tasks. +- **Auto metas:** When `auto` is ON, the LM infers bpm/key/duration/time_signature. When OFF, your manual values are sent directly. +- Use **Show Text** nodes connected to `info` outputs to inspect what the server returned (LLM-generated parameters, generation metadata, etc.). ## License diff --git a/nodes.py b/nodes.py index f9d22f2..4b5aa81 100644 --- a/nodes.py +++ b/nodes.py @@ -299,9 +299,9 @@ class AceStepText2MusicGenParams: }, "optional": { "sample_query": ("STRING", { - "default": "", + "default": "a funk rock song with groovy bass and punchy drums", "multiline": True, - "placeholder": "Describe the music you want (e.g. upbeat pop song about summer)", + "placeholder": "Describe the music you want (e.g. a funk rock song with groovy bass)", "tooltip": "Natural language description for LLM to generate caption/lyrics/metadata", }), "is_instrumental": ("BOOLEAN", {