v1.14.2: Fix NeMo diarization for PyTorch 2.10+, add JSON Extractor/Builder nodes, Qwen3-TTS voice presets & robustness fixes
This commit is contained in:
@@ -1483,6 +1483,39 @@ B: 太神奇了!那我们快去生成试试吧!
|
||||
- **自动归档**: 支持将拼接结果自动保存为 `.txt` 文件,文件名支持**自定义前缀** (save_prefix),方便回溯。
|
||||
- **输出**: `STRING` (拼接后的文本)。
|
||||
|
||||
#### 6.2 JSON 提取器 (AIIA JSON Extractor 🔑)
|
||||
|
||||
- **用途**: 从 `STRING` 类型的 JSON 字符串中,按 key 路径提取指定字段的值。无需依赖第三方 JSON 节点,完全基于 `STRING` 类型,兼容所有上下游节点。
|
||||
- **功能**:
|
||||
- **嵌套路径**: 支持 dot 路径 + 数组索引,例如 `data.items[0].name`、`[2].speaker`。
|
||||
- **多类型输出**: 同时输出 `STRING`、`INT`、`FLOAT`、`BOOLEAN`,自动安全转换。
|
||||
- **鲁棒容错**: JSON 格式错误、key 不存在、类型不匹配等情况**不会崩溃**,统一返回 `fallback` 默认值。
|
||||
- **自动清理**: 自动处理 BOM、首尾空白等常见 JSON 字符串问题。
|
||||
- **输入**:
|
||||
- `json_string` (STRING, forceInput) — 待解析的 JSON 字符串
|
||||
- `key_path` (STRING) — 提取路径,例: `name`, `data.items[0].text`, `[2].speaker`
|
||||
- `fallback` (STRING, 可选) — 解析失败时的默认值
|
||||
- **输出**: `value` (STRING), `value_int` (INT), `value_float` (FLOAT), `found` (BOOLEAN)
|
||||
|
||||
**路径语法示例:**
|
||||
|
||||
| 路径 | JSON 示例 | 提取结果 |
|
||||
|---|---|---|
|
||||
| `name` | `{"name": "Alice"}` | `Alice` |
|
||||
| `data.count` | `{"data": {"count": 42}}` | `42` |
|
||||
| `items[0].text` | `{"items": [{"text": "Hello"}]}` | `Hello` |
|
||||
| `[2].speaker` | `[{}, {}, {"speaker": "B"}]` | `B` |
|
||||
| *(空)* | `{"a": 1}` | 返回整个 JSON |
|
||||
|
||||
#### 6.3 JSON 构建器 (AIIA JSON Builder 🏗️)
|
||||
|
||||
- **用途**: 将多组 key-value 组装为 JSON 对象字符串,方便下游节点消费。
|
||||
- **功能**:
|
||||
- 支持最多 **4 对** key-value 输入。
|
||||
- 值如果是合法 JSON(数组、对象、数字、布尔),会自动解析为对应类型而非字符串。
|
||||
- **输入**: `key_1` + `value_1` (必填), `key_2`~`key_4` + `value_2`~`value_4` (可选)
|
||||
- **输出**: `json_string` (STRING)
|
||||
|
||||
---
|
||||
|
||||
## ❓ 故障排查
|
||||
@@ -1500,6 +1533,18 @@ B: 太神奇了!那我们快去生成试试吧!
|
||||
|
||||
## Changelog
|
||||
|
||||
### [1.14.2] - 2026-02-17
|
||||
|
||||
- **NeMo Diarization 兼容性修复**: 修复 PyTorch 2.10+ 环境下 NeMo 说话人分割(`diarize()`)因 lhotse 1.32 不兼容报 `object.__init__() takes exactly one argument` 的崩溃。
|
||||
- 自动检测 PyTorch 版本,仅在 >= 2.10 时 monkey-patch `CutSampler.__init__`,移除已废弃的 `data_source` 参数。
|
||||
- **JSON Extractor 节点** (New): 从 `STRING` 类型的 JSON 字符串中按 key 路径提取值,支持嵌套路径和数组索引(如 `data.items[0].name`)。
|
||||
- 多类型输出:`STRING`、`INT`、`FLOAT`、`BOOLEAN`。
|
||||
- 鲁棒容错:JSON 格式错误、key 不存在时返回 `fallback` 默认值,不会崩溃。
|
||||
- **JSON Builder 节点** (New): 将多组 key-value 组装为 JSON 对象字符串,支持最多 4 对输入。
|
||||
- **Qwen3-TTS Voice Preset**: 新增 `voice_preset` 下拉框(Female_HQ/Male_HQ/Female/Male),无需手动接入参考音频即可使用内置音色,自动启用 Zero-Shot 模式。
|
||||
- **Qwen3-TTS 鲁棒性**: `reference_text` 为空或字面值 `"None"` 时自动切换到 Zero-Shot 模式,防止意外走入慢速 ICL 模式。
|
||||
- **CosyVoice 参考音频限制**: 参考音频超过 30 秒时自动截断,防止 `AssertionError`。
|
||||
|
||||
### [1.12.0] - 2026-02-16
|
||||
|
||||
- **Podcast Stitcher - MMS Forced Alignment 集成**: 新增 `use_forced_align` 开关,使用 Facebook MMS 声学模型(~1.2GB)进行字级强制对齐。
|
||||
|
||||
@@ -154,6 +154,9 @@ else:
|
||||
# 31. 处理 aiia_text_splitter.py (单人文本拆分)
|
||||
_load_nodes_from_module(".aiia_text_splitter", "aiia_text_splitter")
|
||||
|
||||
# 32. 处理 aiia_json_extractor.py (JSON 提取/构建)
|
||||
_load_nodes_from_module(".aiia_json_extractor", "aiia_json_extractor")
|
||||
|
||||
# 告诉 ComfyUI 这个节点包有一个包含网页资源的 'js' 目录
|
||||
WEB_DIRECTORY = "js"
|
||||
|
||||
|
||||
@@ -836,6 +836,12 @@ class AIIA_CosyVoice_TTS:
|
||||
ref_wav = torchaudio.transforms.Resample(reference_audio["sample_rate"], sample_rate)(ref_wav)
|
||||
|
||||
if ref_wav.abs().max() > 1.0: ref_wav = ref_wav / ref_wav.abs().max()
|
||||
|
||||
# Auto-truncate to 30s (CosyVoice _extract_speech_token limit)
|
||||
max_ref_samples = sample_rate * 30
|
||||
if ref_wav.shape[-1] > max_ref_samples:
|
||||
print(f"[AIIA] Reference audio is {ref_wav.shape[-1]/sample_rate:.1f}s, auto-truncating to 30s.")
|
||||
ref_wav = ref_wav[..., :max_ref_samples]
|
||||
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp_ref:
|
||||
ref_path = tmp_ref.name
|
||||
|
||||
@@ -7,6 +7,47 @@ from omegaconf import OmegaConf, open_dict # open_dict 允许添加新键
|
||||
import folder_paths # ComfyUI 的路径管理模块
|
||||
from typing import Optional, List, Dict, Union # 确保导入类型提示
|
||||
|
||||
# --- Fix: lhotse 1.32 / PyTorch 2.10+ 兼容性补丁 ---
|
||||
# PyTorch 2.10 移除了 Sampler.__init__(data_source=...) 参数,
|
||||
# 但 lhotse 的 CutSampler 仍然传递它,导致:
|
||||
# TypeError: object.__init__() takes exactly one argument
|
||||
# 在这里进行一次性修复,使 NeMo diarization 能正常工作。
|
||||
try:
|
||||
from torch.utils.data import Sampler as _TorchSampler
|
||||
# 检测 Sampler 是否已经不接受 data_source (PyTorch >= 2.10)
|
||||
import inspect
|
||||
_sampler_params = inspect.signature(_TorchSampler.__init__).parameters
|
||||
if 'data_source' not in _sampler_params:
|
||||
try:
|
||||
from lhotse.dataset.sampling.base import CutSampler as _LhotseCutSampler
|
||||
_original_init = _LhotseCutSampler.__init__
|
||||
|
||||
def _patched_cut_sampler_init(self, *args, **kwargs):
|
||||
# 绕过 super().__init__(data_source=None),直接初始化 Sampler
|
||||
_TorchSampler.__init__(self)
|
||||
# 执行 CutSampler 自身的初始化逻辑
|
||||
self.drop_last = kwargs.get('drop_last', False)
|
||||
self.shuffle = kwargs.get('shuffle', False)
|
||||
self.seed = kwargs.get('seed', 0)
|
||||
self.epoch = 0
|
||||
from lhotse.dataset.sampling.base import SamplingDiagnostics, _filter_nothing
|
||||
self._diagnostics = SamplingDiagnostics()
|
||||
self._just_restored_state = False
|
||||
self._maybe_init_distributed(
|
||||
world_size=kwargs.get('world_size', None),
|
||||
rank=kwargs.get('rank', None)
|
||||
)
|
||||
from lhotse.cut import Cut
|
||||
self._filter_fn = _filter_nothing()
|
||||
self._transforms = []
|
||||
|
||||
_LhotseCutSampler.__init__ = _patched_cut_sampler_init
|
||||
print("[AIIA] ✅ 已修补 lhotse CutSampler 以兼容 PyTorch 2.10+")
|
||||
except ImportError:
|
||||
pass # lhotse 未安装,无需修补
|
||||
except Exception as _patch_err:
|
||||
print(f"[AIIA] ⚠️ lhotse 兼容性补丁失败: {_patch_err}")
|
||||
|
||||
# --- 全局模型路径定义 ---
|
||||
_NEMO_MODELS_SUBDIR_STR = "nemo_models"
|
||||
_E2E_MODEL_FILENAME_MAP = { # 支持多种E2E模型
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
"""
|
||||
AIIA JSON Extractor — 从 STRING 中提取 JSON 某个 key 的值
|
||||
|
||||
支持嵌套 key 路径 (e.g. "data.items[0].name"),
|
||||
异常输入时返回 fallback 值而非崩溃。
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
|
||||
class AIIA_JSON_Extractor:
|
||||
"""从 JSON 字符串中按 key 路径提取值,返回 STRING。"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"json_string": ("STRING", {"forceInput": True}),
|
||||
"key_path": ("STRING", {
|
||||
"default": "",
|
||||
"tooltip": "提取路径,支持嵌套和数组索引。例: name, data.items[0].text, [2].speaker"
|
||||
}),
|
||||
},
|
||||
"optional": {
|
||||
"fallback": ("STRING", {
|
||||
"default": "",
|
||||
"tooltip": "解析失败或 key 不存在时返回的默认值"
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "INT", "FLOAT", "BOOLEAN",)
|
||||
RETURN_NAMES = ("value", "value_int", "value_float", "found",)
|
||||
FUNCTION = "extract"
|
||||
CATEGORY = "AIIA/Utils"
|
||||
|
||||
@staticmethod
|
||||
def _parse_path(key_path):
|
||||
"""
|
||||
Parse a dot-separated key path with optional array indices.
|
||||
Examples:
|
||||
"name" -> ["name"]
|
||||
"data.items[0]" -> ["data", "items", 0]
|
||||
"[2].speaker" -> [2, "speaker"]
|
||||
"lines[0].text" -> ["lines", 0, "text"]
|
||||
"""
|
||||
if not key_path or not key_path.strip():
|
||||
return []
|
||||
|
||||
tokens = []
|
||||
# Split by '.' first, then handle [N] inside each part
|
||||
for part in key_path.strip().split('.'):
|
||||
if not part:
|
||||
continue
|
||||
# Match segments like "items[0]" or "[3]"
|
||||
sub_parts = re.findall(r'([^\[\]]+)|\[(\d+)\]', part)
|
||||
for name, idx in sub_parts:
|
||||
if idx != '':
|
||||
tokens.append(int(idx))
|
||||
elif name:
|
||||
tokens.append(name)
|
||||
return tokens
|
||||
|
||||
@staticmethod
|
||||
def _navigate(data, tokens):
|
||||
"""Walk into data following tokens. Returns (value, found)."""
|
||||
current = data
|
||||
for token in tokens:
|
||||
try:
|
||||
if isinstance(token, int):
|
||||
if isinstance(current, (list, tuple)) and -len(current) <= token < len(current):
|
||||
current = current[token]
|
||||
else:
|
||||
return None, False
|
||||
elif isinstance(current, dict):
|
||||
if token in current:
|
||||
current = current[token]
|
||||
else:
|
||||
return None, False
|
||||
else:
|
||||
return None, False
|
||||
except (KeyError, IndexError, TypeError):
|
||||
return None, False
|
||||
return current, True
|
||||
|
||||
def extract(self, json_string, key_path, fallback=""):
|
||||
log = "[AIIA JSON Extractor]"
|
||||
|
||||
# --- 1. Parse JSON ---
|
||||
data = None
|
||||
try:
|
||||
data = json.loads(json_string)
|
||||
except (json.JSONDecodeError, TypeError) as e:
|
||||
print(f"{log} JSON 解析失败: {e}")
|
||||
# Try to recover: strip leading/trailing whitespace, BOM, etc.
|
||||
if isinstance(json_string, str):
|
||||
cleaned = json_string.strip().lstrip('\ufeff')
|
||||
try:
|
||||
data = json.loads(cleaned)
|
||||
print(f"{log} 清理后解析成功")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if data is None:
|
||||
print(f"{log} 无法解析 JSON,返回 fallback: {fallback[:50]}")
|
||||
return (fallback, 0, 0.0, False)
|
||||
|
||||
# --- 2. Navigate key path ---
|
||||
tokens = self._parse_path(key_path)
|
||||
|
||||
if not tokens:
|
||||
# No key path: return the entire JSON re-serialized
|
||||
if isinstance(data, (dict, list)):
|
||||
result_str = json.dumps(data, ensure_ascii=False, indent=2)
|
||||
else:
|
||||
result_str = str(data)
|
||||
return (result_str, self._safe_int(data), self._safe_float(data), True)
|
||||
|
||||
value, found = self._navigate(data, tokens)
|
||||
|
||||
if not found:
|
||||
print(f"{log} key '{key_path}' 未找到,返回 fallback")
|
||||
return (fallback, 0, 0.0, False)
|
||||
|
||||
# --- 3. Convert to output types ---
|
||||
if isinstance(value, (dict, list)):
|
||||
value_str = json.dumps(value, ensure_ascii=False, indent=2)
|
||||
elif value is None:
|
||||
value_str = ""
|
||||
elif isinstance(value, bool):
|
||||
value_str = str(value).lower()
|
||||
else:
|
||||
value_str = str(value)
|
||||
|
||||
return (value_str, self._safe_int(value), self._safe_float(value), True)
|
||||
|
||||
@staticmethod
|
||||
def _safe_int(value):
|
||||
try:
|
||||
if isinstance(value, bool):
|
||||
return 1 if value else 0
|
||||
return int(value)
|
||||
except (ValueError, TypeError):
|
||||
return 0
|
||||
|
||||
@staticmethod
|
||||
def _safe_float(value):
|
||||
try:
|
||||
if isinstance(value, bool):
|
||||
return 1.0 if value else 0.0
|
||||
return float(value)
|
||||
except (ValueError, TypeError):
|
||||
return 0.0
|
||||
|
||||
|
||||
class AIIA_JSON_Builder:
|
||||
"""将多个 STRING 输入组装为 JSON 对象字符串。"""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"key_1": ("STRING", {"default": "key1"}),
|
||||
"value_1": ("STRING", {"default": "", "forceInput": True}),
|
||||
},
|
||||
"optional": {
|
||||
"key_2": ("STRING", {"default": ""}),
|
||||
"value_2": ("STRING", {"default": "", "forceInput": True}),
|
||||
"key_3": ("STRING", {"default": ""}),
|
||||
"value_3": ("STRING", {"default": "", "forceInput": True}),
|
||||
"key_4": ("STRING", {"default": ""}),
|
||||
"value_4": ("STRING", {"default": "", "forceInput": True}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
RETURN_NAMES = ("json_string",)
|
||||
FUNCTION = "build"
|
||||
CATEGORY = "AIIA/Utils"
|
||||
|
||||
def build(self, key_1, value_1, **kwargs):
|
||||
result = {}
|
||||
pairs = [(key_1, value_1)]
|
||||
for i in range(2, 5):
|
||||
k = kwargs.get(f"key_{i}", "")
|
||||
v = kwargs.get(f"value_{i}", "")
|
||||
if k and k.strip():
|
||||
pairs.append((k.strip(), v))
|
||||
|
||||
for key, val in pairs:
|
||||
# Try to parse value as JSON (for nested objects/arrays/numbers/bools)
|
||||
try:
|
||||
parsed = json.loads(val)
|
||||
result[key] = parsed
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
result[key] = val
|
||||
|
||||
return (json.dumps(result, ensure_ascii=False, indent=2),)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"AIIA_JSON_Extractor": AIIA_JSON_Extractor,
|
||||
"AIIA_JSON_Builder": AIIA_JSON_Builder,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"AIIA_JSON_Extractor": "AIIA JSON Extractor 🔑",
|
||||
"AIIA_JSON_Builder": "AIIA JSON Builder 🏗️",
|
||||
}
|
||||
+24
-1
@@ -262,6 +262,7 @@ class AIIA_Qwen_TTS:
|
||||
},
|
||||
"optional": {
|
||||
"speaker": (QWEN_SPEAKER_LIST, {"default": "Vivian"}),
|
||||
"voice_preset": (["None", "Female_HQ", "Male_HQ", "Female", "Male"], {"default": "None", "tooltip": "Built-in voice preset for Base/Clone models. Used when no reference_audio is connected. Auto-enables zero-shot mode."}),
|
||||
"instruct": ("STRING", {"multiline": True, "default": ""}),
|
||||
"reference_audio": ("AUDIO",),
|
||||
"reference_text": ("STRING", {"multiline": True, "default": ""}),
|
||||
@@ -290,6 +291,7 @@ class AIIA_Qwen_TTS:
|
||||
reference_audio = kwargs.get("reference_audio", None)
|
||||
reference_text = kwargs.get("reference_text", "")
|
||||
zero_shot_mode = kwargs.get("zero_shot_mode", False)
|
||||
voice_preset = kwargs.get("voice_preset", "None")
|
||||
emotion = kwargs.get("emotion", "None")
|
||||
dialect = kwargs.get("dialect", "None")
|
||||
seed = kwargs.get("seed", 42)
|
||||
@@ -299,6 +301,27 @@ class AIIA_Qwen_TTS:
|
||||
top_k = kwargs.get("top_k", 20)
|
||||
top_p = kwargs.get("top_p", 0.95)
|
||||
|
||||
# --- Voice Preset Fallback: auto-load seed audio when no reference_audio ---
|
||||
if reference_audio is None and voice_preset != "None":
|
||||
import torchaudio
|
||||
preset_map = {
|
||||
"Female_HQ": "seed_female_hq.wav",
|
||||
"Male_HQ": "seed_male_hq.wav",
|
||||
"Female": "seed_female.wav",
|
||||
"Male": "seed_male.wav",
|
||||
}
|
||||
assets_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "assets")
|
||||
preset_path = os.path.join(assets_dir, preset_map.get(voice_preset, "seed_female_hq.wav"))
|
||||
if os.path.exists(preset_path):
|
||||
wav, sr = torchaudio.load(preset_path)
|
||||
if wav.shape[0] > 1:
|
||||
wav = torch.mean(wav, dim=0, keepdim=True)
|
||||
reference_audio = {"waveform": wav.unsqueeze(0), "sample_rate": sr}
|
||||
zero_shot_mode = True # Preset seeds have no matching text
|
||||
print(f"[AIIA] Qwen3-TTS: Using voice preset '{voice_preset}' as reference audio (zero-shot mode).")
|
||||
else:
|
||||
print(f"[AIIA Warning] Voice preset file not found: {preset_path}")
|
||||
|
||||
# 0. Handle Bundle Routing
|
||||
active_qwen = qwen_model
|
||||
if qwen_model.get("is_bundle"):
|
||||
@@ -485,7 +508,7 @@ class AIIA_Qwen_TTS:
|
||||
|
||||
ref_audio_data = (ref_wav.squeeze().cpu().numpy(), ref_sr)
|
||||
|
||||
ref_text = reference_text if reference_text and reference_text.strip() != "" else None
|
||||
ref_text = reference_text if reference_text and reference_text.strip() not in ("", "None", "none") else None
|
||||
mode_param = zero_shot_mode
|
||||
|
||||
# Robustness: Qwen requires ref_text for ICL mode (zero_shot_mode=False)
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "aiia"
|
||||
description = "Advanced AI Audio/Video toolkit for ComfyUI. Features Multi-Role Podcast/Dialogue Generation, High-Fidelity Voice Cloning (CosyVoice/VibeVoice), TTS, Media Management, and efficient Video tools."
|
||||
version = "1.14.1"
|
||||
version = "1.14.2"
|
||||
license = {file = "LICENSE"}
|
||||
readme = "README.md"
|
||||
authors = [
|
||||
|
||||
Reference in New Issue
Block a user