feat(audio): pronunciation table for GPT-SoVITS characters

"Pronunciation" on a character card edits the node's `replace` setting
(script text -> what the character reads). Each row can be heard as the
character reads it now (the saved table applies) and as replaced, through
a Preview Audio run that stays out of the audio gallery; one preview at a
time, cancelled when the dialog closes. The file is re-read before editing
and every other settings field is kept. buildTtsPrompt gains `preview`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DemonGatanjieu
2026-09-29 02:41:44 +08:00
co-authored by Claude Opus 5.5
parent e341b8a26d
commit 6cb9ea39be
9 changed files with 291 additions and 9 deletions
+4
View File
@@ -237,6 +237,10 @@ covered by the `styles.css` manifest.
character's language, speed and folded sampling parameters to its `defaults`
(only values that differ from the node's own defaults, which it reads from
`/object_info`; needs Anomalous_TTS interface 11).
`ui_tts_pronunciation.js` edits a GPT-SoVITS character's pronunciation table
(the node's `replace` setting) from its card; each row can be heard as it reads
now and as replaced, through a Preview Audio run, and saving keeps every other
settings field.
`audio_tts_run.js` queues one API prompt with this page's client id and
follows it (websocket status, `/history` result, cancel = queue delete or
targeted `/interrupt`); it never touches the canvas.
+1
View File
@@ -5,6 +5,7 @@
### 🎙️ Audio & Voice Studio (音频与声音工作台)
- **Voice library for ComfyUI-F5-TTS (F5-TTS 声音库)**: follows F5-TTS's own layout — `Character.wav` is the main voice and `Character.<emotion>.wav` are its emotion variants (used as `{emotion}` tags). Voices can be previewed, uploaded with their reference text (with one-click romanization for Japanese/Korean text), and dragged onto a TTS node; the node's own path spelling is used on every OS.
- **Generate right in the Script Director (剧本台直接生成)**: for GPT-SoVITS characters, **Generate** runs the script in ComfyUI without touching the canvas (it waits in the queue while ComfyUI is busy), plays the result and saves it to `output/audio/<character>/`. **↻** on a line card gives just that line another take while the others come from cache; **New take of all** starts over with a new seed. Language, speed and the sampling parameters (folded under **Advanced**, with a hint on each) start from the character's saved defaults; when a take sounds right, **Save these settings as this character's default** keeps them for next time. The script drawer is wider and its footer scrolls, so the line cards keep room. Needs Anomalous_TTS interface 11.
- **Pronunciation table (读音表)**: **Pronunciation** on a GPT-SoVITS character card lists what the script says and what the character should read instead (`C站 → 西站`); each line can be heard as the character reads it now and as replaced, without filling the audio gallery.
- **Audio page tabs (音频页顶部标签)**: the "Audio Workflows" tab is gone from the audio page; generating now happens in the Script Director, and saved workflows stay under **Workflows** on the image side.
- **Script Director (剧本台)**: paste a script for one character; each sentence becomes a card with emotion chips, inline editing, split at cursor, merge and delete. The finished script is pushed to the selected F5-TTS node or dragged onto one, with the tags written for you.
- **Audio gallery (音频库)**: lists generated audio with its speech text, voice and seed read from the file itself, supports search, and can save unsaved previews into `output/audio/`.
+8
View File
@@ -124,6 +124,14 @@ GPT-SoVITS 角色在剧本台里多一块 **生成** 区:
需要 Anomalous_TTS 接口版本 11 以上;旧版本时这里会提示更新,仍然可以推送到节点、在画布上运行。
### GPT-SoVITS:某个词总是读错
角色卡片上的 **读音** 打开这个角色的读音表:左边写剧本里的字,右边写让它实际读的字,例如 `C站 → 西站`、`粘进 → 沾进`。剧本和字幕照常写,只影响念出来的声音。
- 每一行的 **现在** 听这个角色现在怎么读左边的字,**改后** 听右边的字,对比着改。试听不会存进音频库。
- 日语模型说中文时特别有用:它发不好的音(选、全这类 ü,模型、行这类后鼻音)换成好读的同音字,或在前面加个字;英文字母写成读音相近的汉字。
- 长的先匹配,只替换一遍。只对这个角色生效。
## 4. 生成和找回结果
- F5-TTS:把剧本推送到画布上的节点后运行。GPT-SoVITS 也可以在剧本台里直接生成(见上)。
+6 -6
View File
@@ -143,19 +143,19 @@ export function ttsOutputPrefix(character) {
export const TTS_NODE_CLASS = 'AnomalousTTS_CharacterSpeech';
/**
* ComfyUI API prompt for one GPT-SoVITS script: the character node, then Save Audio.
* ComfyUI API prompt for one GPT-SoVITS script: the character node, then Save Audio
* (`preview: true`: Preview Audio, a temp file that stays out of the audio gallery).
* `inputs` are more node inputs by name (speed, top_k …); left out, the node's defaults apply.
*/
export function buildTtsPrompt({ character, speech, seed, language = 'auto', ...inputs }) {
export function buildTtsPrompt({ character, speech, seed, language = 'auto', preview = false, ...inputs }) {
return {
1: {
class_type: TTS_NODE_CLASS,
inputs: { character, text: speech, seed, language, ...inputs },
},
2: {
class_type: 'SaveAudio',
inputs: { audio: ['1', 0], filename_prefix: ttsOutputPrefix(character) },
},
2: preview
? { class_type: 'PreviewAudio', inputs: { audio: ['1', 0] } }
: { class_type: 'SaveAudio', inputs: { audio: ['1', 0], filename_prefix: ttsOutputPrefix(character) } },
};
}
+28
View File
@@ -1333,6 +1333,20 @@ export const i18n = {
audioDragCardHintGptSovits: "拖标题栏 → GPT-SoVITS 节点 · 点 {标签} 复制",
audioDragCharacterTitleGptSovits: "拖到 GPT-SoVITS 节点:角色设为 {character}。情绪在台词里用 {标签} 切换。",
audioDropReject_otherEngine: "这是 {voiceEngine} 的声音,{node} 节点用不了,请拖到 {voiceEngine} 的节点上",
ttsPronounceOpen: '读音',
ttsPronounceTitle: '读音:{name}',
ttsPronounceSubtitle: '剧本里写左边的字,这个角色就读右边的字。剧本和字幕照常写,只影响念出来的声音',
ttsPronounceTips: '长的先匹配,只替换一遍。日语模型说中文时:发不好的音(选、全这类 ü,模型、行这类后鼻音)换成好读的同音字,或在前面加个字;英文字母写成读音相近的汉字(C站 → 西站);多音字读错就换成同音字(粘进 → 沾进)。右边留空表示不读。',
ttsPronounceLanguage: '试听语言',
ttsPronounceFromPlaceholder: '剧本里写的',
ttsPronounceToPlaceholder: '让角色读成',
ttsPronounceNow: '现在',
ttsPronounceNowHint: '听这个角色现在怎么读左边的字(已保存的读音表会生效)',
ttsPronounceAfter: '改后',
ttsPronounceAfterHint: '听右边的字读出来是什么样',
ttsPronounceAdd: '+ 添加一条',
ttsPronounceFromMissing: '有一条只写了读法({to}),左边要写剧本里的字',
ttsPronounceDuplicate: '“{from}”写了两遍,请合并成一条',
ttsEditorOpen: "编辑情绪",
ttsEditorTitle: "编辑情绪 · {name}",
ttsEditorSubtitle: "为主声音和每种情绪各挑一条参考音频(3~10 秒、有台词最好)。保存后写进角色文件夹里的 anomalous_tts.json。",
@@ -3108,6 +3122,20 @@ export const i18n = {
audioDragCardHintGptSovits: "Drag the header → GPT-SoVITS node · click a {tag} to copy",
audioDragCharacterTitleGptSovits: "Drop onto a GPT-SoVITS node to set its character to {character}. Switch emotions with {tags} in the text.",
audioDropReject_otherEngine: "This is a {voiceEngine} voice; the {node} node cannot use it. Drop it on the {voiceEngine} node",
ttsPronounceOpen: 'Pronunciation',
ttsPronounceTitle: 'Pronunciation: {name}',
ttsPronounceSubtitle: 'When the script says the left text, this character reads the right text. Scripts and subtitles stay as written; only the voice changes',
ttsPronounceTips: 'Longest match first, replaced once. A Japanese model speaking Chinese: swap sounds it cannot say (ü as in 选, -ng as in 模型) for easier same-sounding characters, or put a character before the word; write Latin letters as Chinese characters that sound alike (C站 → 西站); fix a misread polyphone with a same-sounding character (粘进 → 沾进). Leave the right side empty to skip the text.',
ttsPronounceLanguage: 'Preview language',
ttsPronounceFromPlaceholder: 'Text in the script',
ttsPronounceToPlaceholder: 'Read it as',
ttsPronounceNow: 'Now',
ttsPronounceNowHint: 'Hear how this character reads the left text now (the saved table applies)',
ttsPronounceAfter: 'After',
ttsPronounceAfterHint: 'Hear the right text read aloud',
ttsPronounceAdd: '+ Add a line',
ttsPronounceFromMissing: 'A line has only a reading ({to}); write the script text on the left',
ttsPronounceDuplicate: '"{from}" is listed twice; keep one line',
ttsEditorOpen: "Edit emotions",
ttsEditorTitle: "Edit emotions · {name}",
ttsEditorSubtitle: "Pick one reference clip (3–10 s, with its line) for the main voice and each emotion. Saved to anomalous_tts.json in the character folder.",
+5 -1
View File
@@ -6,6 +6,7 @@ import { getActiveAudioFilter, renderAudioSidebar, setActiveAudioFilter } from '
import { bindVoiceDrag } from './audio_voice_drag.js';
import { AUDIO_ENGINES, detectEngines, engineById, engineTargetLabels, getStoredEngine, invalidateEngineCache, loadEngine, loadGptSovitsStatus, pickEngine, setStoredEngine } from './audio_engines.js';
import { openGptSovitsEditor } from './ui_audio_tts_editor.js';
import { openPronunciationEditor } from './ui_tts_pronunciation.js';
import { bindTtsFileDrop } from './ui_tts_file_drop.js';
import { openTtsImport } from './ui_tts_import.js';
import {
@@ -39,6 +40,7 @@ const SVG = {
SEARCH: `<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="11" cy="11" r="8"/><line x1="21" y1="21" x2="16.65" y2="16.65"/></svg>`,
SCRIPT: `<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 19.5v-15A2.5 2.5 0 0 1 6.5 2H20v20H6.5a2.5 2.5 0 0 1 0-5H20"/></svg>`,
PLUS: `<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="12" y1="5" x2="12" y2="19"/><line x1="5" y1="12" x2="19" y2="12"/></svg>`,
PRONOUNCE: `<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 7V5h10v2"/><path d="M9 5v14"/><path d="M7 19h4"/><path d="M15 13h6"/><path d="M18 10v9"/></svg>`,
EDIT: `<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 20h9"/><path d="M16.5 3.5a2.1 2.1 0 0 1 3 3L7 19l-4 1 1-4Z"/></svg>`,
REFRESH: `<svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12a9 9 0 1 1-2.64-6.36"/><polyline points="21 3 21 9 15 9"/></svg>`,
};
@@ -251,7 +253,9 @@ function renderCharacterCard(group, owner, { onChanged, canImport = false } = {}
if (isTts && group.raw && !group.raw.error) {
const edit = createToolButton(SVG.EDIT, t('ttsEditorOpen'), 'is-compact');
edit.onclick = () => openGptSovitsEditor(group, { onSaved: () => onChanged?.() });
headerRight.append(edit);
const pronounce = createToolButton(SVG.PRONOUNCE, t('ttsPronounceOpen'), 'is-compact');
pronounce.onclick = () => openPronunciationEditor(group, { onSaved: () => onChanged?.() });
headerRight.append(edit, pronounce);
if (canImport) {
const add = createToolButton(SVG.PLUS, t('ttsImportAddOpen'), 'is-compact');
add.onclick = () => openTtsImport({ target: group.character, onDone: () => onChanged?.() });
+201
View File
@@ -0,0 +1,201 @@
import { t } from './interface_settings.js';
import { createViewScope } from './ui_lifecycle.js';
import { anomalousAlert } from './ui_dialog.js';
import { buildTtsPrompt } from './audio_script.js';
import { startPromptJob } from './audio_tts_run.js';
import { fetchGptSovitsCharacter, saveGptSovitsSettings } from './audio_engines.js';
/**
* GPT-SoVITS pronunciation table: "what the script says -> what this character
* reads" (the node's `replace` setting, applied before synthesis). Each row can
* be heard both ways through a short Preview Audio run (audio_tts_run.js):
* "now" sends the original through the node, so the saved table applies; "after"
* sends the replacement. Saving keeps every other settings field.
*/
let activeScope = null;
const LANGUAGES = ['auto', 'zh', 'ja', 'en'];
function el(tag, className, text) {
const node = document.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
function button(className, label, onClick, title) {
const btn = el('button', className, label);
btn.type = 'button';
if (title) { btn.title = title; btn.setAttribute('aria-label', title); }
btn.onclick = onClick;
return btn;
}
function textInput(className, value, placeholder) {
const input = el('input', className);
input.type = 'text';
input.placeholder = placeholder;
input.value = value || '';
return input;
}
/** First problem as a message, or '' when the rows can be saved. */
export function validatePronunciationRows(rows) {
const seen = new Set();
for (const row of rows) {
if (!row.from.trim()) return t('ttsPronounceFromMissing', { to: row.to });
if (seen.has(row.from)) return t('ttsPronounceDuplicate', { from: row.from });
seen.add(row.from);
}
return '';
}
/**
* Open the table for one GPT-SoVITS voice group (audio_engines.gptSovitsGroups shape).
* `onSaved(group)` receives the refreshed group after the node accepted the settings.
*/
export function openPronunciationEditor(group, { onSaved } = {}) {
activeScope?.dispose();
const scope = createViewScope();
activeScope = scope;
const raw = group.raw || {};
const name = raw.name || group.character;
let settings = raw.settings && typeof raw.settings === 'object' ? raw.settings : {};
let loaded = false;
let job = null;
const player = new Audio();
const seed = Math.floor(Math.random() * 2 ** 31);
const overlay = el('div', 'anomalous-voice-modal-overlay');
const modal = el('div', 'anomalous-voice-modal anomalous-tts-editor anomalous-tts-pronounce');
modal.setAttribute('role', 'dialog');
modal.setAttribute('aria-modal', 'true');
const close = () => scope.dispose();
scope.onDispose(() => {
job?.cancel();
player.pause();
overlay.remove();
if (activeScope === scope) activeScope = null;
});
overlay.onclick = (e) => { if (e.target === overlay) close(); };
scope.listen(window, 'keydown', (e) => {
if (e.key !== 'Escape' || document.querySelector('.anomalous-dialog-overlay')) return;
e.stopPropagation();
close();
}, true);
const header = el('div', 'anomalous-voice-modal-header');
const heading = el('div', 'anomalous-voice-modal-heading');
heading.append(
el('div', 'anomalous-voice-modal-title', t('ttsPronounceTitle', { name })),
el('div', 'anomalous-voice-modal-subtitle', t('ttsPronounceSubtitle')),
);
header.append(heading, button('anomalous-voice-modal-close', '×', close, t('close')));
const languageBar = el('label', 'anomalous-tts-pronounce-language');
const language = el('select', 'anomalous-sd-run-select');
for (const code of LANGUAGES) {
const option = el('option', '', t(`scriptRunLang_${code}`));
option.value = code;
language.appendChild(option);
}
language.value = LANGUAGES.includes(settings.defaults?.language) ? settings.defaults.language : 'auto';
languageBar.append(el('span', '', t('ttsPronounceLanguage')), language);
/** Play `text` through the node (Preview Audio); one preview at a time. */
const preview = async (text, btn) => {
if (!text.trim() || job) return;
player.pause();
const label = btn.textContent;
btn.textContent = '…';
btn.classList.add('is-busy');
job = startPromptJob(buildTtsPrompt({ character: name, speech: text.trim(), seed, language: language.value, preview: true }));
try {
const { outputs } = await job.result;
const audio = outputs['2']?.audio?.[0];
if (!audio || scope.signal.aborted) return;
const query = new URLSearchParams({ filename: audio.filename, subfolder: audio.subfolder || '', type: audio.type || 'temp' });
player.src = `/view?${query}&t=${Date.now()}`;
player.play().catch(() => {});
} catch (error) {
if (!error.cancelled && !scope.signal.aborted) await anomalousAlert(t('scriptRunFailed', { error: error.message }));
} finally {
job = null;
btn.textContent = label;
btn.classList.remove('is-busy');
}
};
const rowsBox = el('div', 'anomalous-tts-rows');
const addRow = (from = '', to = '') => {
const line = el('div', 'anomalous-tts-row');
const fromInput = textInput('anomalous-tts-name-input', from, t('ttsPronounceFromPlaceholder'));
const toInput = textInput('anomalous-tts-text-input', to, t('ttsPronounceToPlaceholder'));
const nowBtn = button('anomalous-tts-listen', t('ttsPronounceNow'), () => preview(fromInput.value, nowBtn), t('ttsPronounceNowHint'));
const afterBtn = button('anomalous-tts-listen', t('ttsPronounceAfter'), () => preview(toInput.value, afterBtn), t('ttsPronounceAfterHint'));
line.append(fromInput, el('span', 'anomalous-tts-arrow', '→'), toInput, nowBtn, afterBtn,
button('anomalous-tts-remove', '×', () => line.remove(), t('ttsEditorRemove')));
rowsBox.appendChild(line);
return fromInput;
};
const fillRows = () => {
rowsBox.replaceChildren();
for (const [from, to] of Object.entries(settings.replace || {})) addRow(from, to);
if (!rowsBox.children.length) addRow();
};
fillRows();
const addBtn = button('anomalous-tts-add', t('ttsPronounceAdd'), () => addRow().focus());
const footer = el('div', 'anomalous-voice-modal-footer');
const saveBtn = button('anomalous-voice-modal-submit', t('ttsEditorSave'), async () => {
if (!loaded) return;
const rows = [...rowsBox.querySelectorAll('.anomalous-tts-row')].map(line => {
const [fromInput, toInput] = line.querySelectorAll('input');
return { from: fromInput.value, to: toInput.value.trim() };
}).filter(row => row.from.trim() || row.to);
const problem = validatePronunciationRows(rows);
if (problem) {
await anomalousAlert(problem);
return;
}
const next = { ...settings, format: 1 };
if (rows.length) next.replace = Object.fromEntries(rows.map(row => [row.from, row.to]));
else delete next.replace;
saveBtn.disabled = true;
try {
const updated = await saveGptSovitsSettings(name, next);
onSaved?.(updated);
close();
} catch (e) {
if (!scope.signal.aborted) await anomalousAlert(t('ttsEditorSaveFailed', { error: e.message }));
} finally {
if (!scope.signal.aborted) saveBtn.disabled = !loaded;
}
});
footer.append(button('anomalous-voice-modal-cancel', t('dialogCancel'), close), saveBtn);
// The summary list may be up to a minute old: read the file as it is now before editing it.
const status = el('div', 'anomalous-tts-hint anomalous-tts-status', t('ttsEditorLoadingFiles'));
saveBtn.disabled = true;
fetchGptSovitsCharacter(name, scope.signal).then(detail => {
if (scope.signal.aborted) return;
if (detail.settings_error) throw new Error(detail.settings_error);
if (detail.settings && typeof detail.settings === 'object') settings = detail.settings;
fillRows();
status.hidden = true;
loaded = true;
saveBtn.disabled = false;
}).catch(e => {
if (scope.signal.aborted) return;
status.textContent = t('ttsEditorLoadFailed', { error: e.message || String(e) });
status.classList.add('is-error');
});
const body = el('div', 'anomalous-tts-editor-body');
body.append(status, el('div', 'anomalous-tts-hint', t('ttsPronounceTips')), languageBar, rowsBox, addBtn);
modal.append(header, body, footer);
overlay.appendChild(modal);
document.body.appendChild(overlay);
rowsBox.querySelector('input')?.focus();
return close;
}
+1 -1
View File
@@ -9,7 +9,7 @@
@import url("./styles/08-theme-gallery-overrides.css?v=20260929-audio-tabs");
@import url("./styles/09-prompt-drawers.css?v=20260918-model-sources-1");
@import url("./styles/10-model-sources.css?v=20260918-model-sources-1");
@import url("./styles/11-audio-studio.css?v=20260926-audio-sidebar-1");
@import url("./styles/11-audio-studio.css?v=20260929-pronunciation");
@import url("./styles/12-tts-setup.css?v=20260926-tts-import-batch-2");
@import url("./styles/13-script-director.css?v=20260929-generate-roomier");
@import url("./styles/14-version-manager.css");
+37 -1
View File
@@ -1571,9 +1571,45 @@
display: none;
}
/* Pronunciation table (ui_tts_pronunciation.js): from → to, two listen buttons, remove */
.anomalous-tts-pronounce .anomalous-tts-row {
grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr) auto auto auto;
}
.anomalous-tts-arrow {
color: #64748b;
font-size: 12px;
}
.anomalous-tts-listen {
border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 6px;
background: rgba(255, 255, 255, 0.04);
color: #cbd5e1;
cursor: pointer;
font-size: 11px;
padding: 4px 8px;
min-width: 40px;
}
.anomalous-tts-listen:hover,
.anomalous-tts-listen.is-busy {
border-color: rgba(56, 189, 248, 0.4);
color: #38bdf8;
}
.anomalous-tts-pronounce-language {
display: flex;
align-items: center;
gap: 8px;
font-size: 12px;
color: #cbd5e1;
}
@media (max-width: 560px) {
.anomalous-tts-row,
.anomalous-tts-row.is-main {
.anomalous-tts-row.is-main,
.anomalous-tts-pronounce .anomalous-tts-row {
grid-template-columns: 1fr auto;
}
}