diff --git a/README.md b/README.md index 236ef9c..cf31a51 100644 --- a/README.md +++ b/README.md @@ -211,6 +211,66 @@ Nodes that are fully self-contained say so in their own docstring. --- +### 🗃️ Repository layout + +One feature per folder; the root only keeps the assembly entry points: + +```text +XB_ToolBox/ +├─ __init__.py node registration + route wiring +├─ prestartup_script.py startup cleanup (ComfyUI convention: must stay at root) +├─ nodes/ node implementations +│ ├─ core/ parameter bus / relay / label / string / switch / accel +│ ├─ image/ prompt presets / encoders / scale / segmentation / comic +│ ├─ video/ Wan / LTX / Hailuo / digital human / video I-O +│ ├─ audio/ audio slicer / YuE2 / play sound +│ ├─ llm/ XB-llama / GGUF loaders +│ ├─ hardware/ VRAM & memory monitoring +│ └─ cosyvoice3/ CosyVoice3 node package +├─ libs/ node-free logic (xb_paths / xb_presets / xb_gallery / xb_skills / yue2_* …) +├─ jzl_workflows/ "JZL Workflows" (data + routes + icon/style) +├─ web/ frontend assets (formerly js/) +├─ locales/ docs/ tests/ i18n / docs / tests +├─ tools/ one-off dev scripts (not used at runtime) +├─ support_llama/ xb_skill/ video_formats/ data folders +└─ LibreHardwareMonitorLib.dll +``` + +--- + +### 🗂️ JZL Workflows (sidebar entry) + +Once XB_ToolBox is installed, ComfyUI's sidebar gains a **"机智罗 / JZL Workflows"** entry, +sitting right above the built-in **Templates** tab. Clicking it opens a **centered dialog** just +like the official Templates browser — category list on the left, search on top, thumbnail cards, +and a card click opens the workflow in a new workflow tab (`Esc` / backdrop click / ✕ closes it). +No ComfyUI frontend files are patched, so frontend upgrades won't break it. + +Everything for this feature lives in **`jzl_workflows/`**: + +```text +XB_ToolBox/ + jzl_workflows/ + __init__.py registration entry (root __init__.py calls it) + routes.py routes: index / workflow / preview / icon+css assets + data/ + index.json English index + index.zh.json Chinese index (same shape as comfyui-workflow-templates) + anima_t2i_basic.json the workflow itself + anima_t2i_basic-1.webp preview image; a placeholder card is shown if missing + web/ + xb_workflow_gallery.css + xb_icon_dark.png / xb_icon_light.png + web/jzl_workflows/ + xb_workflow_gallery.js panel script (must stay under web/ to be loaded) +``` + +To add a workflow: drop the JSON into `jzl_workflows/data/` (ASCII file name), add an entry to +both index files (`name` must match the file name), and put the preview next to it as +`-1.webp`. + +--- + ### 🙏 Credits XB_ToolBox stands on the shoulders of the ComfyUI ecosystem. Several nodes are ports, wrappers or adaptations — thanks to their authors: diff --git a/README_zh.md b/README_zh.md index 714923c..6b1ddee 100644 --- a/README_zh.md +++ b/README_zh.md @@ -210,6 +210,65 @@ pip install -r requirements.txt --- +### 🗃️ 目录结构 + +按「一个功能一个目录」组织，根目录只留装配入口： + +```text +XB_ToolBox/ +├─ __init__.py 节点注册 + 路由装配 +├─ prestartup_script.py ComfyUI 启动清理（约定：必须留在根） +├─ nodes/ 节点实现 +│ ├─ core/ 参数总线 / 接力 / 标签 / 字符串 / 开关 / 加速 +│ ├─ image/ 提示词预设 / 编码 / 缩放 / 分割 / 漫画 / 神笔 +│ ├─ video/ Wan / LTX / Hailuo / 数字人 / 视频读写 +│ ├─ audio/ 音频切片 / YuE2 / 播放声音 +│ ├─ llm/ XB-llama / GGUF 加载 +│ ├─ hardware/ 显存与内存监控 +│ └─ cosyvoice3/ CosyVoice3 节点包 +├─ libs/ 无节点的纯逻辑（xb_paths / xb_presets / xb_gallery / xb_skills / yue2_* …） +├─ jzl_workflows/ 「机智罗的工作流」（数据 + 路由 + 图标样式） +├─ web/ 前端资源（原 js/） +├─ locales/ docs/ tests/ 国际化 / 文档 / 测试 +├─ tools/ 开发用一次性脚本（不参与运行） +├─ support_llama/ xb_skill/ video_formats/ 数据目录 +└─ LibreHardwareMonitorLib.dll +``` + +--- + +### 🗂️ 机智罗的工作流（侧栏入口） + +装上 XB_ToolBox 后，ComfyUI 左侧栏会多出一个 **「机智罗」** 入口（紧贴官方 **「模板」** 上方）， +点一下就会像官方「模板」那样在**屏幕正中弹出大面板**：左侧分类、顶部搜索、缩略图卡片网格， +点卡片直接在工作流新标签页打开，`Esc` / 点遮罩 / 右上角 ✕ 收起。 +不需要改任何 ComfyUI 前端文件，升级 ComfyUI 也不会失效。 + +工作流数据都在 **`jzl_workflows/`** 一个目录里，方便单独管理： + +```text +XB_ToolBox/ + jzl_workflows/ + __init__.py 注册入口（根 __init__.py 只调用它） + routes.py 后端路由：清单 / 工作流 / 预览图 / 图标样式 + data/ + index.json 英文清单 + index.zh.json 中文清单（结构同官方 comfyui-workflow-templates） + anima_t2i_basic.json 工作流本身 + anima_t2i_basic-1.webp 同名预览图，缺省时显示占位卡片 + web/ + xb_workflow_gallery.css + xb_icon_dark.png / xb_icon_light.png + web/jzl_workflows/ + xb_workflow_gallery.js 面板脚本（必须在 web/ 下 ComfyUI 才会加载） +``` + +新增工作流只要两步：把工作流 JSON 放进 `jzl_workflows/data/`（文件名用 ASCII）， +再在 `data/index.json`、`data/index.zh.json` 里各加一条条目（`name` 与 JSON 文件名一致）。 +预览图放同目录，命名 `-1.webp`。 + +--- + ### 🙏 致谢 XB_ToolBox 站在 ComfyUI 生态的肩膀上，部分节点为移植 / 套壳 / 改编，感谢原作者： diff --git a/__init__.py b/__init__.py index 8ba04c3..94a10ba 100644 --- a/__init__.py +++ b/__init__.py @@ -28,8 +28,8 @@ async def choose_folder(request): return web.json_response({"path": "", "error": f"弹窗调用失败: {str(e)}"}) -from .nodes_audio_slicer import handle_audio_waveform -from .nodes_model_loader_v1 import _filter_by_keyword +from .nodes.audio.nodes_audio_slicer import handle_audio_waveform +from .nodes.llm.nodes_model_loader_v1 import _filter_by_keyword @PromptServer.instance.routes.post("/xb_toolbox/audio_waveform") async def get_audio_waveform(request): @@ -39,7 +39,7 @@ async def get_audio_waveform(request): @PromptServer.instance.routes.get("/xb_toolbox/yue2_recipes") async def yue2_recipes(request): """YuE2 风格配方表：前端「YuE2 风格提示」节点联动用，与 yue2_styles.RECIPES 同一数据源""" - from .yue2_styles import GENRES, RECIPES + from .libs.yue2_styles import GENRES, RECIPES return web.json_response({"recipes": RECIPES, "genres": GENRES}) @@ -61,6 +61,14 @@ async def get_model_list(request): }) +# ── 「机智罗的工作流」模板数据服务（清单 + 工作流 + 预览图 + 图标样式）── +try: + from .jzl_workflows import register_routes as _register_jzl_workflow_routes + _register_jzl_workflow_routes() +except Exception as _xb_tpl_err: + print(f"[XB_ToolBox] 工作流模板服务注册失败: {_xb_tpl_err}") + + def print_success(msg): print(f"\033[92m{msg}\033[0m") @@ -82,42 +90,42 @@ XB_LOGO = "\n".join([ NODE_CLASS_MAPPINGS = {} NODE_DISPLAY_NAME_MAPPINGS = {} -WEB_DIRECTORY = os.path.join(os.path.dirname(os.path.abspath(__file__)), "js") +WEB_DIRECTORY = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web") try: - from .nodes_vis import XB_VRAM_Calculator, XB_ChunkVisualization - from .nodes_vram import XTX_Data_Radar - from .nodes_video import XB_VideoParamsMaster, XB_ImageParamsMaster, XB_ImageParamsMasterMini, XB_MasterParameter, XB_VideoLoader, XB_VideoCombine - from .nodes_blockswap import XB_UNetBlockSwap, XB_CheckpointBlockSwap - from .nodes_blockswap_sage import XB_Sage_BlockSwap - from .nodes_wiring import XB_UNetNameBroadcaster, XB_CLIPNameBroadcaster - from .nodes_memory_viz import XB_MemoryVisualization - from .nodes_wan_vae import (XB_WanImageToVideo, XB_WanFirstLastFrameToVideo, XB_WanSoundImageToVideo, XB_WanFunControlToVideo, XB_WanVaceToVideo, XB_Wan22FunControlToVideo, XB_WanInfiniteTalkToVideo, XB_WanInfiniteTalkToVideo_Single, XB_WanInfiniteTalkToVideo_Dual, XB_WanFunInpaintToVideo, XB_WanCameraImageToVideo, XB_WanPhantomSubjectToVideo, XB_WanHuMoImageToVideo, XB_Wan22ImageToVideoLatent, XB_WanSoundImageToVideoExtend, XB_WanSCAILToVideo, XB_WanSCAILToVideoPro, XB_WanDancerVideo, XB_WanDanceSwitcher, XB_WanDancerCombo, XB_BerniniConditioning) - from .nodes_batch import XB_BatchFolderLoader, XB_BatchFolderBatchLoader - from .nodes_pipeline import XB_Wan_ParamBus, XB_Wan_RelayNode, XB_Wan_InfiniteRelayNode, XB_Video_Merger, XB_StoryboardSlicer,XB_WanAnimate_ParamBus,XB_WanAnimate_RelayNode, XB_WanInfiniteTalk_ParamBus, XB_WanInfiniteTalk_RelayNode, XB_Wan_InfiniteRelayNode_New, XB_WanAnimate_RelayNode_New, XB_WanInfiniteTalk_RelayNode_New, XB_WanSCAIL_ParamBus_New, XB_WanSCAIL_RelayNode_New, XB_WanInfiniteTalk_RelayNode_MultiRef, XB_WanInfiniteTalk_RelayNode_AllInOne - from .nodes_ltx_streamer import XB_LTX23_InfiniteStreamer - from .nodes_sageatt import XB_SageAttentionAccelerator - from .nodes_wan import XB_WanCompileSettings, XB_WanAnimateToVideo + from .nodes.hardware.nodes_vis import XB_VRAM_Calculator, XB_ChunkVisualization + from .nodes.hardware.nodes_vram import XTX_Data_Radar + from .nodes.video.nodes_video import XB_VideoParamsMaster, XB_ImageParamsMaster, XB_ImageParamsMasterMini, XB_MasterParameter, XB_VideoLoader, XB_VideoCombine + from .nodes.core.nodes_blockswap import XB_UNetBlockSwap, XB_CheckpointBlockSwap + from .nodes.core.nodes_blockswap_sage import XB_Sage_BlockSwap + from .nodes.core.nodes_wiring import XB_UNetNameBroadcaster, XB_CLIPNameBroadcaster + from .nodes.hardware.nodes_memory_viz import XB_MemoryVisualization + from .nodes.video.nodes_wan_vae import (XB_WanImageToVideo, XB_WanFirstLastFrameToVideo, XB_WanSoundImageToVideo, XB_WanFunControlToVideo, XB_WanVaceToVideo, XB_Wan22FunControlToVideo, XB_WanInfiniteTalkToVideo, XB_WanInfiniteTalkToVideo_Single, XB_WanInfiniteTalkToVideo_Dual, XB_WanFunInpaintToVideo, XB_WanCameraImageToVideo, XB_WanPhantomSubjectToVideo, XB_WanHuMoImageToVideo, XB_Wan22ImageToVideoLatent, XB_WanSoundImageToVideoExtend, XB_WanSCAILToVideo, XB_WanSCAILToVideoPro, XB_WanDancerVideo, XB_WanDanceSwitcher, XB_WanDancerCombo, XB_BerniniConditioning) + from .nodes.core.nodes_batch import XB_BatchFolderLoader, XB_BatchFolderBatchLoader + from .nodes.video.nodes_pipeline import XB_Wan_ParamBus, XB_Wan_RelayNode, XB_Wan_InfiniteRelayNode, XB_Video_Merger, XB_StoryboardSlicer,XB_WanAnimate_ParamBus,XB_WanAnimate_RelayNode, XB_WanInfiniteTalk_ParamBus, XB_WanInfiniteTalk_RelayNode, XB_Wan_InfiniteRelayNode_New, XB_WanAnimate_RelayNode_New, XB_WanInfiniteTalk_RelayNode_New, XB_WanSCAIL_ParamBus_New, XB_WanSCAIL_RelayNode_New, XB_WanInfiniteTalk_RelayNode_MultiRef, XB_WanInfiniteTalk_RelayNode_AllInOne + from .nodes.video.nodes_ltx_streamer import XB_LTX23_InfiniteStreamer + from .nodes.core.nodes_sageatt import XB_SageAttentionAccelerator + from .nodes.video.nodes_wan import XB_WanCompileSettings, XB_WanAnimateToVideo - from .nodes_label import XB_CanvasLabel - from .nodes_audio_slicer import XB_AudioSlicer, XB_AudioSlicerV1, XB_AudioSlicerV2, XB_AudioSlicerV3 - from .nodes_digital_human import XB_DigitalHumanParams_Single, XB_DigitalHumanParams_Dual - from .nodes_model_loader_gguf import XB_ModelLoaderV1_GGUF, XB_ModelLoaderV2_GGUF, XB_ModelLoaderV3_GGUF - from .nodes_status_switch import XB_NodeStatusSwitch - from .nodes_fast_toggle import XB_DisableGroups, XB_DisableNodes, XB_AnySwitch - from .nodes_flux_encode import XB_FluxMultiImageEncode - from .nodes_qwen_encode import XB_TextEncodeQwenImageEdit, XB_TextEncodeQwenImageEditPlus - from .nodes_image_scale import XB_ImageScale, XB_ImageScalePixels - from .nodes_shenbi import XB_ShenBi - from .nodes_hailuo_video import XB_HailuoH3VideoParams - from .nodes_list_dispatcher import XB_ListDispatcher - from .nodes_batch_images import XB_BatchImages - from .nodes_reference_any import XB_ReferenceAny - from .nodes_segmentation import XB_HumanSegModelLoader, XB_HumanSegmentation + from .nodes.core.nodes_label import XB_CanvasLabel + from .nodes.audio.nodes_audio_slicer import XB_AudioSlicer, XB_AudioSlicerV1, XB_AudioSlicerV2, XB_AudioSlicerV3 + from .nodes.video.nodes_digital_human import XB_DigitalHumanParams_Single, XB_DigitalHumanParams_Dual + from .nodes.llm.nodes_model_loader_gguf import XB_ModelLoaderV1_GGUF, XB_ModelLoaderV2_GGUF, XB_ModelLoaderV3_GGUF + from .nodes.core.nodes_status_switch import XB_NodeStatusSwitch + from .nodes.core.nodes_fast_toggle import XB_DisableGroups, XB_DisableNodes, XB_AnySwitch + from .nodes.image.nodes_flux_encode import XB_FluxMultiImageEncode + from .nodes.image.nodes_qwen_encode import XB_TextEncodeQwenImageEdit, XB_TextEncodeQwenImageEditPlus + from .nodes.image.nodes_image_scale import XB_ImageScale, XB_ImageScalePixels + from .nodes.image.nodes_shenbi import XB_ShenBi + from .nodes.video.nodes_hailuo_video import XB_HailuoH3VideoParams + from .nodes.core.nodes_list_dispatcher import XB_ListDispatcher + from .nodes.core.nodes_batch_images import XB_BatchImages + from .nodes.core.nodes_reference_any import XB_ReferenceAny + from .nodes.image.nodes_segmentation import XB_HumanSegModelLoader, XB_HumanSegmentation # 注册视频转码端点（预览窗口实时缩放依赖） - from . import xb_video_server - from .nodes_vanilla_wrappers import (XB_KSampler, XB_KSamplerAdvanced, + from .libs import xb_video_server + from .nodes.core.nodes_vanilla_wrappers import (XB_KSampler, XB_KSamplerAdvanced, XB_SamplerCustom, XB_SamplerCustomAdvanced, XB_VAEDecode, XB_VAEDecodeTiled, XB_VAEDecodeTiledImage, @@ -125,16 +133,16 @@ try: XB_VAEEncodeTiledImage, XB_VAEEncodeForInpaint, _AliasVAEDecodeTemporal) - from .nodes_string_merge import XB_StringMerge - from .nodes_clip_text_edit import XB_CLIPTextEdit - from .nodes_lora_loader import XB_TripleLoraLoader, XB_TripleLoraLoaderModelOnly - from .nodes_msr import XB_MSR - from .nodes_comic import XB_ComicPromptParser, XB_ComicTextRenderer, XB_AutoBubbleTextRenderer - from .nodes_super import XB_BerniniPromptRouter, XB_K2StyleRouter + from .nodes.core.nodes_string_merge import XB_StringMerge + from .nodes.core.nodes_clip_text_edit import XB_CLIPTextEdit + from .nodes.core.nodes_lora_loader import XB_TripleLoraLoader, XB_TripleLoraLoaderModelOnly + from .nodes.core.nodes_msr import XB_MSR + from .nodes.image.nodes_comic import XB_ComicPromptParser, XB_ComicTextRenderer, XB_AutoBubbleTextRenderer + from .nodes.core.nodes_super import XB_BerniniPromptRouter, XB_K2StyleRouter # ── Llama 节点 (LLM/VLM) ── try: - from .nodes_llama import ( + from .nodes.llm.nodes_llama import ( XB_llamaModelLoader, XB_llamaModelLoaderPro, XB_llamaInstruct, XB_llamaParameters, XB_llamaUnloadModel, XB_llamaCleanStates, XB_llamaBBoxes2BBox, XB_llamaUnpackCodeBlock, XB_llamaPromptEnhancer, @@ -144,7 +152,7 @@ try: XB_RoleSceneDispatcher, ) # ── 提示词增强反推（模型加载器 + 指令推理 + 提示词增强预设 + 推理参数 + LLM-API 设置 五合一）── - from .nodes_llama_prompt_reverse import XB_llamaPromptReverse + from .nodes.llm.nodes_llama_prompt_reverse import XB_llamaPromptReverse _LLAMA_AVAILABLE = True except ImportError as e: print_warning(f"[XB-llama] llama-cpp-python 未安装, Llama/MiniMax 节点不可用: {e}") @@ -152,7 +160,7 @@ try: # ── 🖼️ 生图提示词预设Pro（生图提示词预设 + 提示词增强反推 融合）── try: - from .nodes_image_prompt_preset_pro import XB_ImagePromptPresetPro + from .nodes.image.nodes_image_prompt_preset_pro import XB_ImagePromptPresetPro _PRO_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] 生图提示词预设Pro 未加载：{e}") @@ -160,7 +168,7 @@ try: # ── 🖼️ Qwen2.1提示词预设（生图提示词预设 + 官方 Load CLIP + 官方 Generate Text）── try: - from .nodes_qwen_prompt_preset import XB_QwenPromptPreset + from .nodes.image.nodes_qwen_prompt_preset import XB_QwenPromptPreset _QWEN_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] Qwen2.1提示词预设 未加载：{e}") @@ -168,30 +176,30 @@ try: # ── 🎵 YuE2 音乐节点（风格提示 / 歌词 / 乐谱工作台 / 翻唱准备 / 转谱导入）── try: - from .nodes_yue2_style import XB_YuE2StylePrompt - from .nodes_yue2_lyrics import XB_YuE2Lyrics - from .nodes_yue2_score import XB_YuE2ScoreWorkbench, XB_YuE2CoverPrep - from .nodes_yue2_transcription import XB_YuE2Transcription + from .nodes.audio.nodes_yue2_style import XB_YuE2StylePrompt + from .nodes.audio.nodes_yue2_lyrics import XB_YuE2Lyrics + from .nodes.audio.nodes_yue2_score import XB_YuE2ScoreWorkbench, XB_YuE2CoverPrep + from .nodes.audio.nodes_yue2_transcription import XB_YuE2Transcription _YUE2_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] YuE2 音乐节点 未加载：{e}") _YUE2_AVAILABLE = False # ── CosyVoice3 音频节点 ── - from .cosyvoice3.nodes.model_loader import XB_CosyVoice3_ModelLoader - from .cosyvoice3.nodes.zero_shot import XB_CosyVoice3_ZeroShot - from .cosyvoice3.nodes.cross_lingual import XB_CosyVoice3_CrossLingual - from .cosyvoice3.nodes.voice_conversion import XB_CosyVoice3_VoiceConversion - from .cosyvoice3.nodes.audio_crop import XB_CosyVoice3_AudioCrop - from .cosyvoice3.nodes.dialog import XB_CosyVoice3_Dialog - from .cosyvoice3.nodes.instruct2 import XB_CosyVoice3_Instruct2 - from .cosyvoice3.nodes.save_speaker import XB_CosyVoice3_SaveSpeaker - from .cosyvoice3.nodes.speaker_clone import XB_CosyVoice3_SpeakerClone - from .cosyvoice3.nodes.speaker_instruct2 import XB_CosyVoice3_SpeakerInstruct2 + from .nodes.cosyvoice3.nodes.model_loader import XB_CosyVoice3_ModelLoader + from .nodes.cosyvoice3.nodes.zero_shot import XB_CosyVoice3_ZeroShot + from .nodes.cosyvoice3.nodes.cross_lingual import XB_CosyVoice3_CrossLingual + from .nodes.cosyvoice3.nodes.voice_conversion import XB_CosyVoice3_VoiceConversion + from .nodes.cosyvoice3.nodes.audio_crop import XB_CosyVoice3_AudioCrop + from .nodes.cosyvoice3.nodes.dialog import XB_CosyVoice3_Dialog + from .nodes.cosyvoice3.nodes.instruct2 import XB_CosyVoice3_Instruct2 + from .nodes.cosyvoice3.nodes.save_speaker import XB_CosyVoice3_SaveSpeaker + from .nodes.cosyvoice3.nodes.speaker_clone import XB_CosyVoice3_SpeakerClone + from .nodes.cosyvoice3.nodes.speaker_instruct2 import XB_CosyVoice3_SpeakerInstruct2 # ── 🎬 保存视频（Create Video + Save Video 二合一，依赖新版 V3 API）── try: - from .nodes_save_video import XB_SaveVideo + from .nodes.video.nodes_save_video import XB_SaveVideo _SAVE_VIDEO_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] 保存视频节点 未加载：{e}") @@ -199,27 +207,27 @@ try: # ── 🎬 加载视频（Load Video + Get Video Components 二合一，依赖新版 V3 API）── try: - from .nodes_load_video import XB_LoadVideo + from .nodes.video.nodes_load_video import XB_LoadVideo _LOAD_VIDEO_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] 加载视频节点 未加载：{e}") _LOAD_VIDEO_AVAILABLE = False # ── 生图提示词预设（提示词 + 空latent 一体化）── - from .nodes_image_prompt_preset import XB_ImagePromptPreset + from .nodes.image.nodes_image_prompt_preset import XB_ImagePromptPreset # ── 生图预设（生图提示词预设 + 姿态参考图 / 穿搭参考图）── # 同时 import xb_gallery → 注册 /xb_toolbox/gallery/* 端点（图库绑定、素材清单、图片读取） try: - from . import xb_gallery # noqa: F401 - from .nodes_image_preset import XB_ImagePreset + from .libs import xb_gallery # noqa: F401 + from .nodes.image.nodes_image_preset import XB_ImagePreset _IMAGE_PRESET_AVAILABLE = True except Exception as e: print_warning(f"[XB-BOX] 生图预设节点 未加载：{e}") _IMAGE_PRESET_AVAILABLE = False # ── 播放声音（复刻 pysssss PlaySound，自带音频资源）── - from .nodes_play_sound import XB_PlaySound + from .nodes.audio.nodes_play_sound import XB_PlaySound NODE_CLASS_MAPPINGS = { "XB_VRAM_Calculator": XB_VRAM_Calculator, diff --git a/jzl_workflows/__init__.py b/jzl_workflows/__init__.py new file mode 100644 index 0000000..f8f8264 --- /dev/null +++ b/jzl_workflows/__init__.py @@ -0,0 +1,17 @@ +"""「机智罗的工作流」— 侧栏入口 + 居中弹窗的模板库 + +全部实现集中在 ``jzl_workflows/`` 一个目录里： + + jzl_workflows/ + __init__.py 本文件：对外只暴露 register_routes() + routes.py 后端路由（清单 / 工作流 / 预览图 / 图标与样式） + data/ index*.json 清单 + 工作流 JSON + 预览图 + web/ 面板样式与深浅两套图标 + +前端脚本只有 ``js/jzl_workflows/xb_workflow_gallery.js`` 留在 ``js/`` 下， +因为 ComfyUI 只加载 ``WEB_DIRECTORY`` 内的 js（该目录会递归扫描子文件夹）。 +""" + +from .routes import register_routes + +__all__ = ["register_routes"] diff --git a/jzl_workflows/data/anima_control_t2i.json b/jzl_workflows/data/anima_control_t2i.json new file mode 100644 index 0000000..8c55cf3 --- /dev/null +++ b/jzl_workflows/data/anima_control_t2i.json @@ -0,0 +1 @@ +{"id":"44f5adba-dfc2-4485-9322-43d74d470679","revision":0,"last_node_id":111,"last_link_id":230,"nodes":[{"id":7,"type":"XB_CanvasLabel","pos":[3400,30],"size":[500,80],"flags":{"allow_interaction":false,"pinned":true},"order":0,"mode":0,"inputs":[],"outputs":[],"title":"更多教程全网搜：机智罗_LX","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":37,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":9,"type":"XB_CanvasLabel","pos":[1180,-260],"size":[2740,300],"flags":{"allow_interaction":false,"pinned":true},"order":1,"mode":0,"inputs":[],"outputs":[],"title":"Anima 局部重绘工作流","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":268,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":67,"type":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","pos":[2480,130],"size":[420,1030],"flags":{"pinned":true},"order":7,"mode":0,"inputs":[{"name":"filename_prefix","type":"STRING","widget":{"name":"filename_prefix"},"link":null},{"name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":null},{"name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":null},{"name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":null},{"name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":null},{"name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":null},{"name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":null},{"name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":null},{"name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":null},{"name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":null},{"label":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":null},{"label":"随机种","name":"seed_1","type":"INT","widget":{"name":"seed_1"},"link":null},{"label":"采样步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":null},{"name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":null},{"label":"分块尺寸","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":null},{"label":"重叠尺寸","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":null},{"label":"解码前执行","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":null},{"name":"name","type":"COMBO","widget":{"name":"name"},"link":null},{"name":"strength","type":"FLOAT","widget":{"name":"strength"},"link":null},{"name":"start_percent","type":"FLOAT","widget":{"name":"start_percent"},"link":null},{"name":"end_percent","type":"FLOAT","widget":{"name":"end_percent"},"link":null},{"label":"提示词","name":"text","type":"STRING","widget":{"name":"text"},"link":null},{"name":"image","type":"IMAGE","link":216},{"name":"mask","shape":7,"type":"MASK","link":217},{"name":"width","type":"INT","widget":{"name":"width"},"link":222},{"name":"height","type":"INT","widget":{"name":"height"},"link":224}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[227]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","previewExposures":[{"name":"$$canvas-image-preview","sourceNodeId":"52","sourcePreviewName":"$$canvas-image-preview"}]},"widgets_values":["图片/Anima/Anima-局部重绘","Anima\\anima-preview3-base.safetensors","Anima\\qwen_3_06b_base.safetensors","Anima\\qwen_image_vae.safetensors","Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1,"关闭",939522274253971,8,1,1920,64,"不做任何清理","Anima\\anima-lllite-inpainting-v2.safetensors",1,0,1,"女孩的眼睛改成红色",1024,1024],"widgets_values_named":{"filename_prefix":"图片/Anima/Anima-局部重绘","unet_name":"Anima\\anima-preview3-base.safetensors","clip_name":"Anima\\qwen_3_06b_base.safetensors","vae_name":"Anima\\qwen_image_vae.safetensors","lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1,"preset":"关闭","seed_1":939522274253971,"steps":8,"cfg":1,"tile_size":1920,"overlap":64,"cleanup":"不做任何清理","name":"Anima\\anima-lllite-inpainting-v2.safetensors","strength":1,"start_percent":0,"end_percent":1,"text":"女孩的眼睛改成红色","width":1024,"height":1024},"color":"#232","bgcolor":"#353"},{"id":104,"type":"LoadImage","pos":[1200,130],"size":[720,710],"flags":{"pinned":true},"order":2,"mode":0,"inputs":[{"localized_name":"图像","name":"image","type":"COMBO","widget":{"name":"image"},"link":null},{"localized_name":"选择文件上传","name":"upload","type":"IMAGEUPLOAD","widget":{"name":"upload"},"link":null}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[215,228]},{"localized_name":"遮罩","name":"MASK","type":"MASK","links":[]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"LoadImage"},"widgets_values":["goat_horned_spirit_girl.png","image"],"widgets_values_named":{"image":"goat_horned_spirit_girl.png","upload":"image"}},{"id":105,"type":"Painter","pos":[1970,130],"size":[460,1030],"flags":{"pinned":true},"order":5,"mode":0,"inputs":[{"localized_name":"图像","name":"image","shape":7,"type":"IMAGE","link":215},{"localized_name":"mask","name":"mask","type":"STRING","widget":{"name":"mask"},"link":null},{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":null},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":null},{"localized_name":"背景颜色","name":"bg_color","type":"COLOR","widget":{"name":"bg_color"},"link":null}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[216,230]},{"localized_name":"遮罩","name":"MASK","type":"MASK","links":[217]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"Painter","painterTool":"brush","painterBrushSize":69,"painterBrushColor":"#000000","painterBrushOpacity":1,"painterBrushHardness":1},"widgets_values":["painter-105-1793813754736.png [input]",1024,1024,"#000000"],"widgets_values_named":{"mask":"painter-105-1793813754736.png [input]","width":1024,"height":1024,"bg_color":"#000000"}},{"id":107,"type":"XB_ImageParamsMasterMini","pos":[1200,900],"size":[720,260],"flags":{"pinned":true},"order":3,"mode":0,"inputs":[{"localized_name":"画幅比例","name":"aspect_ratio","type":"COMBO","widget":{"name":"aspect_ratio"},"link":null},{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":null},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":null},{"localized_name":"生成数量","name":"batch_size","type":"INT","widget":{"name":"batch_size"},"link":null},{"localized_name":"倍数","name":"multiple","type":"INT","widget":{"name":"multiple"},"link":null}],"outputs":[{"localized_name":"图片宽度","name":"Image Width","type":"INT","links":[222]},{"localized_name":"图片高度","name":"Image Height","type":"INT","links":[224]},{"localized_name":"生成数量","name":"Batch Size","type":"INT","links":null},{"localized_name":"缩放尺寸","name":"Scale Size","type":"INT","links":null}],"properties":{"cnr_id":"XB_ToolBox","ver":"21eb17f086bbb64708a0716e2efeb03fa25bfba2","Node name for S&R":"XB_ImageParamsMasterMini"},"widgets_values":["自定义",1024,1024,1,16],"widgets_values_named":{"aspect_ratio":"自定义","width":1024,"height":1024,"batch_size":1,"multiple":16},"color":"#16727c","bgcolor":"#4F0074"},{"id":108,"type":"ImageCompare","pos":[2950,130],"size":[930,1030],"flags":{"pinned":true},"order":8,"mode":0,"inputs":[{"localized_name":"image_a","name":"image_a","shape":7,"type":"IMAGE","link":227},{"localized_name":"image_b","name":"image_b","shape":7,"type":"IMAGE","link":228},{"localized_name":"compare_view","name":"compare_view","type":"IMAGECOMPARE","widget":{"name":"compare_view"},"link":null}],"outputs":[],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"ImageCompare"},"widgets_values":[],"widgets_values_named":{}},{"id":110,"type":"GetImageSize","pos":[2070,1280],"size":[230,70],"flags":{"collapsed":false},"order":4,"mode":0,"inputs":[{"label":"图像","localized_name":"图像","name":"image","type":"IMAGE","link":null}],"outputs":[{"label":"宽度","localized_name":"宽度","name":"width","type":"INT","links":[]},{"label":"高度","localized_name":"高度","name":"height","type":"INT","links":[]},{"localized_name":"批处理大小","name":"batch_size","type":"INT","links":[]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"GetImageSize"}},{"id":111,"type":"PreviewImage","pos":[2690,1310],"size":[400,300],"flags":{},"order":6,"mode":0,"inputs":[{"localized_name":"图像","name":"images","type":"IMAGE","link":230}],"outputs":[{"localized_name":"images","name":"images","type":"IMAGE","links":null}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"PreviewImage"}}],"links":[[215,104,0,105,0,"IMAGE"],[227,67,0,108,0,"IMAGE"],[228,104,0,108,1,"IMAGE"],[216,105,0,67,22,"IMAGE"],[217,105,1,67,23,"MASK"],[222,107,0,67,24,"INT"],[224,107,1,67,25,"INT"],[230,105,0,111,0,"IMAGE"]],"groups":[{"id":1,"title":"Anima-文生图-机智罗","bounding":[1190,60,2700,1110],"color":"#3f789e","flags":{"pinned":true}}],"definitions":{"subgraphs":[{"id":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","version":1,"state":{"lastGroupId":9,"lastNodeId":111,"lastLinkId":230,"lastRerouteId":0},"revision":0,"config":{},"name":"模型加载+采样+解码","inputNode":{"id":-10,"bounding":[240,1540,135.0234375,568]},"outputNode":{"id":-20,"bounding":[2380,1580,128,68]},"inputs":[{"id":"3cd285bf-894e-4810-879d-f17e59ea3450","name":"filename_prefix","type":"STRING","linkIds":[229],"pos":[351.0234375,2064]},{"id":"c4cef3a7-2176-4c21-b780-338df3e1e01b","name":"unet_name","type":"COMBO","linkIds":[146],"pos":[351.0234375,1564]},{"id":"cb8d4371-1883-4f71-865c-84d6da7805b8","name":"clip_name","type":"COMBO","linkIds":[147],"pos":[351.0234375,1584]},{"id":"5e1462bd-1d34-4601-b8d0-810526159ead","name":"vae_name","type":"COMBO","linkIds":[148],"pos":[351.0234375,1604]},{"id":"bcda50ec-08c1-4ca2-8c5a-1a508f31e92f","name":"lora_01","type":"COMBO","linkIds":[149],"pos":[351.0234375,1624]},{"id":"7711dbe8-17f3-4a7c-88f2-dcb8c6087f5b","name":"strength_01","type":"FLOAT","linkIds":[150],"pos":[351.0234375,1644]},{"id":"423da43e-d6e5-441d-9548-aa27f1019e5e","name":"lora_02","type":"COMBO","linkIds":[151],"pos":[351.0234375,1664]},{"id":"ff806aa1-83fe-4271-bfb7-150cde936068","name":"strength_02","type":"FLOAT","linkIds":[152],"pos":[351.0234375,1684]},{"id":"4b26b78f-0a2c-47ac-8793-b37411f94895","name":"lora_03","type":"COMBO","linkIds":[153],"pos":[351.0234375,1704]},{"id":"c2af5451-2693-4f3b-904e-552a90c27946","name":"strength_03","type":"FLOAT","linkIds":[154],"pos":[351.0234375,1724]},{"id":"c68b1c5d-dd69-4b9c-8622-61277812ce2a","name":"preset","type":"COMBO","linkIds":[155],"label":"加速方式","pos":[351.0234375,1744]},{"id":"2f117623-fcc3-4f38-9ecb-e271c51d7f66","name":"seed_1","type":"INT","linkIds":[156],"label":"随机种","pos":[351.0234375,1764]},{"id":"64fbd3d6-b7be-49c3-8a98-3d7348fa1792","name":"steps","type":"INT","linkIds":[157],"label":"采样步数","pos":[351.0234375,1784]},{"id":"19192258-d662-4645-9c3f-4208d6272399","name":"cfg","type":"FLOAT","linkIds":[158],"pos":[351.0234375,1804]},{"id":"8e79a56a-593c-4f95-aa9e-27a51dc2941f","name":"tile_size","type":"INT","linkIds":[201],"label":"分块尺寸","pos":[351.0234375,1824]},{"id":"e24ba591-c57f-43b5-81b7-185ce1f14d21","name":"overlap","type":"INT","linkIds":[202],"label":"重叠尺寸","pos":[351.0234375,1844]},{"id":"4356c50a-2e2c-49ac-8b6d-a26febe29ec0","name":"cleanup","type":"COMBO","linkIds":[203],"label":"解码前执行","pos":[351.0234375,1864]},{"id":"79d52521-67d6-493a-9fb2-f114ca8ae14c","name":"name","type":"COMBO","linkIds":[209],"pos":[351.0234375,1924]},{"id":"90e8ffe4-afe8-4745-bf97-39e1d77de9db","name":"strength","type":"FLOAT","linkIds":[210],"pos":[351.0234375,1944]},{"id":"dd00b1b7-3613-4b82-9a79-2b4c60887e8f","name":"start_percent","type":"FLOAT","linkIds":[211],"pos":[351.0234375,1964]},{"id":"56ebbee1-f7ec-4df5-b950-ff2c7c0d1fe5","name":"end_percent","type":"FLOAT","linkIds":[212],"pos":[351.0234375,1984]},{"id":"9b614adf-fceb-4258-becb-bbaad01a7c59","name":"text","type":"STRING","linkIds":[221],"label":"提示词","pos":[351.0234375,2044]},{"id":"ffd6ddd9-8c05-4dce-8f81-3d2dfcf38f94","name":"image","type":"IMAGE","linkIds":[207],"pos":[351.0234375,1884]},{"id":"df797fd2-96fc-487b-bc50-2421ce883da4","name":"mask","type":"MASK","linkIds":[208],"pos":[351.0234375,1904]},{"id":"309c0c02-9d72-46ea-9f4e-6fe3d6d4533b","name":"width","type":"INT","linkIds":[219],"pos":[351.0234375,2004]},{"id":"3d7c0ea0-9ae1-46d0-8434-da37fcc98ce8","name":"height","type":"INT","linkIds":[220],"pos":[351.0234375,2024]}],"outputs":[{"id":"04f44b9f-158c-4600-bb17-709ca515c335","name":"IMAGE","type":"IMAGE","linkIds":[226],"localized_name":"图像","pos":[2404,1604]}],"widgets":[],"nodes":[{"id":65,"type":"UNETLoader","pos":[510,1190],"size":[440,110],"flags":{},"order":8,"mode":0,"inputs":[{"localized_name":"UNet名称","name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":146},{"localized_name":"数据类型","name":"weight_dtype","type":"COMBO","widget":{"name":"weight_dtype"},"link":null}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[205]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"UNETLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\anima-preview3-base.safetensors","default"],"widgets_values_named":{"unet_name":"Anima\\anima-preview3-base.safetensors","weight_dtype":"default"},"color":"#323","bgcolor":"#535"},{"id":64,"type":"XB_TripleLoraLoaderModelOnly","pos":[510,1350],"size":[440,230],"flags":{},"order":7,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":206},{"localized_name":"LORA-01","name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":149},{"localized_name":"强度-01","name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":150},{"localized_name":"LORA-02","name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":151},{"localized_name":"强度-02","name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":152},{"localized_name":"LORA-03","name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":153},{"localized_name":"强度-03","name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":154}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[133]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_TripleLoraLoaderModelOnly"},"widgets_values":["Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1],"widgets_values_named":{"lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1},"color":"#16727c","bgcolor":"#4F0074"},{"id":51,"type":"CLIPTextEncode","pos":[1080,1180],"size":[340,620],"flags":{},"order":1,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":130},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":221}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","slot_index":0,"links":[102]}],"title":"CLIP Text Encode (Positive Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPTextEncode","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"],"widgets_values_named":{"text":"(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"},"color":"#232","bgcolor":"#353"},{"id":66,"type":"XB_SageAttentionAccelerator","pos":[510,1630],"size":[440,90],"flags":{},"order":9,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":133},{"localized_name":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":155}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[134]}],"properties":{"cnr_id":"XB_ToolBox","ver":"2b291a666b781c0b10117bc51e5edf286ffda5a5","Node name for S&R":"XB_SageAttentionAccelerator","aux_id":"WJLUOXIAO/XB_ToolBox"},"widgets_values":["comfy kitchen attention"],"widgets_values_named":{"preset":"comfy kitchen attention"},"color":"#16727c","bgcolor":"#4F0074"},{"id":63,"type":"CLIPLoader","pos":[510,1770],"size":[440,140],"flags":{},"order":6,"mode":0,"inputs":[{"localized_name":"CLIP名称","name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":147},{"localized_name":"类型","name":"type","type":"COMBO","widget":{"name":"type"},"link":null},{"localized_name":"设备","name":"device","shape":7,"type":"COMBO","widget":{"name":"device"},"link":null}],"outputs":[{"localized_name":"CLIP","name":"CLIP","type":"CLIP","links":[130,131]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_3_06b_base.safetensors","stable_diffusion","default"],"widgets_values_named":{"clip_name":"Anima\\qwen_3_06b_base.safetensors","type":"stable_diffusion","device":"default"},"color":"#323","bgcolor":"#535"},{"id":50,"type":"CLIPTextEncode","pos":[1080,1850],"size":[340,130],"flags":{},"order":0,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":131},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":null}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","links":[103]}],"title":"CLIP Text Encode (Negative Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.3.65","Node name for S&R":"CLIPTextEncode","enableTabs":false,"tabWidth":65,"tabXOffset":10,"hasSecondTab":false,"secondTabText":"Send Back","secondTabOffset":80,"secondTabWidth":65},"widgets_values":["worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"],"widgets_values_named":{"text":"worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"},"color":"#322","bgcolor":"#533"},{"id":62,"type":"VAELoader","pos":[510,1960],"size":[440,90],"flags":{},"order":5,"mode":0,"inputs":[{"localized_name":"vae名称","name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":148}],"outputs":[{"localized_name":"VAE","name":"VAE","type":"VAE","links":[132]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"VAELoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_image_vae.safetensors"],"widgets_values_named":{"vae_name":"Anima\\qwen_image_vae.safetensors"},"color":"#323","bgcolor":"#535"},{"id":52,"type":"KSampler","pos":[1480,1460],"size":[310,410],"flags":{},"order":2,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":134},{"localized_name":"正面条件","name":"positive","type":"CONDITIONING","link":102},{"localized_name":"负面条件","name":"negative","type":"CONDITIONING","link":103},{"localized_name":"Latent图像","name":"latent_image","type":"LATENT","link":218},{"localized_name":"种子","name":"seed","type":"INT","widget":{"name":"seed"},"link":156},{"localized_name":"步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":157},{"localized_name":"cfg","name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":158},{"localized_name":"采样器名称","name":"sampler_name","type":"COMBO","widget":{"name":"sampler_name"},"link":null},{"localized_name":"调度器","name":"scheduler","type":"COMBO","widget":{"name":"scheduler"},"link":null},{"localized_name":"降噪","name":"denoise","type":"FLOAT","widget":{"name":"denoise"},"link":null}],"outputs":[{"localized_name":"Latent","name":"LATENT","type":"LATENT","links":[105]}],"properties":{"cnr_id":"comfy-core","ver":"0.42.1","Node name for S&R":"KSampler"},"widgets_values":[466366138910810,"randomize",8,1,"res_multistep","simple",1],"widgets_values_named":{"seed":466366138910810,"control_after_generate":"randomize","steps":8,"cfg":1,"sampler_name":"res_multistep","scheduler":"simple","denoise":1}},{"id":53,"type":"XB_PlaySound","pos":[1520,1400],"size":[270,110],"flags":{"collapsed":true},"order":3,"mode":0,"inputs":[{"localized_name":"任意输入","name":"any","type":"*","link":104},{"localized_name":"播放模式","name":"mode","type":"COMBO","widget":{"name":"mode"},"link":null},{"localized_name":"音量","name":"volume","type":"FLOAT","widget":{"name":"volume"},"link":null},{"localized_name":"声音文件","name":"file","type":"STRING","widget":{"name":"file"},"link":null}],"outputs":[{"localized_name":"任意输出","name":"any","shape":6,"type":"*","links":[]}],"title":"🔊 播放声音","properties":{"cnr_id":"XB_ToolBox","ver":"7058d1613c09977817f62bf226065ff90d55cb61","Node name for S&R":"XB_PlaySound"},"widgets_values":["always",0.5,"notify.mp3"],"widgets_values_named":{"mode":"always","volume":0.5,"file":"notify.mp3"},"color":"#16727c","bgcolor":"#4F0074"},{"id":54,"type":"XB_VAEDecodeTiledImage","pos":[1480,1180],"size":[310,170],"flags":{},"order":4,"mode":0,"inputs":[{"localized_name":"潜空间","name":"samples","type":"LATENT","link":105},{"localized_name":"VAE","name":"vae","type":"VAE","link":132},{"localized_name":"分块大小","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":201},{"localized_name":"重叠","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":202},{"localized_name":"执行前清理","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":203}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[104,225]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_VAEDecodeTiledImage"},"widgets_values":[1920,64,"不做任何清理"],"widgets_values_named":{"tile_size":1920,"overlap":64,"cleanup":"不做任何清理"},"color":"#16727c","bgcolor":"#4F0074"},{"id":101,"type":"ModelPatchLoader","pos":[510,2100],"size":[440,90],"flags":{},"order":10,"mode":0,"inputs":[{"localized_name":"名称","name":"name","type":"COMBO","widget":{"name":"name"},"link":209}],"outputs":[{"localized_name":"模型补丁","name":"MODEL_PATCH","type":"MODEL_PATCH","links":[204]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"ModelPatchLoader","models":[{"name":"anima-lllite-inpainting-v2.safetensors","url":"https://huggingface.co/Comfy-Org/Anima-LLLite/resolve/main/model_patches/anima-lllite-inpainting-v2.safetensors","directory":"model_patches"}]},"widgets_values":["Anima\\anima-lllite-inpainting-v2.safetensors"],"widgets_values_named":{"name":"Anima\\anima-lllite-inpainting-v2.safetensors"}},{"id":102,"type":"AnimaLLLiteApply","pos":[1000,2090],"size":[270,240],"flags":{},"order":11,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":205},{"localized_name":"模型补丁","name":"model_patch","type":"MODEL_PATCH","link":204},{"localized_name":"图像","name":"image","type":"IMAGE","link":207},{"localized_name":"掩码","name":"mask","shape":7,"type":"MASK","link":208},{"localized_name":"强度","name":"strength","type":"FLOAT","widget":{"name":"strength"},"link":210},{"localized_name":"起始百分比","name":"start_percent","type":"FLOAT","widget":{"name":"start_percent"},"link":211},{"localized_name":"结束百分比","name":"end_percent","type":"FLOAT","widget":{"name":"end_percent"},"link":212}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[206]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"AnimaLLLiteApply"},"widgets_values":[1,0,1],"widgets_values_named":{"strength":1,"start_percent":0,"end_percent":1}},{"id":106,"type":"EmptyLatentImage","pos":[550,2260],"size":[310,150],"flags":{},"order":12,"mode":0,"inputs":[{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":219},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":220},{"localized_name":"批量大小","name":"batch_size","type":"INT","widget":{"name":"batch_size"},"link":null}],"outputs":[{"localized_name":"Latent","name":"LATENT","type":"LATENT","links":[218]}],"properties":{"cnr_id":"comfy-core","ver":"0.3.40","Node name for S&R":"EmptyLatentImage","enableTabs":false,"tabWidth":65,"tabXOffset":10,"hasSecondTab":false,"secondTabText":"Send Back","secondTabOffset":80,"secondTabWidth":65},"widgets_values":[1024,1024,1],"widgets_values_named":{"width":1024,"height":1024,"batch_size":1}},{"id":109,"type":"SaveImage","pos":[1850,1200],"size":[490,620],"flags":{},"order":13,"mode":0,"inputs":[{"localized_name":"图片","name":"images","type":"IMAGE","link":225},{"localized_name":"文件名前缀","name":"filename_prefix","type":"STRING","widget":{"name":"filename_prefix"},"link":229}],"outputs":[{"localized_name":"images","name":"images","type":"IMAGE","links":[226]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"SaveImage","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["图片/Anima/文生图/%date:yyyy-MM-dd%/Anima-文生图"],"widgets_values_named":{"filename_prefix":"图片/Anima/文生图/%date:yyyy-MM-dd%/Anima-文生图"},"color":"#223","bgcolor":"#335"}],"groups":[],"links":[{"id":130,"origin_id":63,"origin_slot":0,"target_id":51,"target_slot":0,"type":"CLIP"},{"id":133,"origin_id":64,"origin_slot":0,"target_id":66,"target_slot":0,"type":"MODEL"},{"id":131,"origin_id":63,"origin_slot":0,"target_id":50,"target_slot":0,"type":"CLIP"},{"id":134,"origin_id":66,"origin_slot":0,"target_id":52,"target_slot":0,"type":"MODEL"},{"id":102,"origin_id":51,"origin_slot":0,"target_id":52,"target_slot":1,"type":"CONDITIONING"},{"id":103,"origin_id":50,"origin_slot":0,"target_id":52,"target_slot":2,"type":"CONDITIONING"},{"id":104,"origin_id":54,"origin_slot":0,"target_id":53,"target_slot":0,"type":"IMAGE"},{"id":105,"origin_id":52,"origin_slot":0,"target_id":54,"target_slot":0,"type":"LATENT"},{"id":132,"origin_id":62,"origin_slot":0,"target_id":54,"target_slot":1,"type":"VAE"},{"id":204,"origin_id":101,"origin_slot":0,"target_id":102,"target_slot":1,"type":"MODEL_PATCH"},{"id":205,"origin_id":65,"origin_slot":0,"target_id":102,"target_slot":0,"type":"MODEL"},{"id":206,"origin_id":102,"origin_slot":0,"target_id":64,"target_slot":0,"type":"MODEL"},{"id":218,"origin_id":106,"origin_slot":0,"target_id":52,"target_slot":3,"type":"LATENT"},{"id":225,"origin_id":54,"origin_slot":0,"target_id":109,"target_slot":0,"type":"IMAGE"},{"id":226,"origin_id":109,"origin_slot":0,"target_id":-20,"target_slot":0,"type":"IMAGE"},{"id":229,"origin_id":-10,"origin_slot":0,"target_id":109,"target_slot":1,"type":"STRING"},{"id":146,"origin_id":-10,"origin_slot":1,"target_id":65,"target_slot":0,"type":"COMBO"},{"id":147,"origin_id":-10,"origin_slot":2,"target_id":63,"target_slot":0,"type":"COMBO"},{"id":148,"origin_id":-10,"origin_slot":3,"target_id":62,"target_slot":0,"type":"COMBO"},{"id":149,"origin_id":-10,"origin_slot":4,"target_id":64,"target_slot":1,"type":"COMBO"},{"id":150,"origin_id":-10,"origin_slot":5,"target_id":64,"target_slot":2,"type":"FLOAT"},{"id":151,"origin_id":-10,"origin_slot":6,"target_id":64,"target_slot":3,"type":"COMBO"},{"id":152,"origin_id":-10,"origin_slot":7,"target_id":64,"target_slot":4,"type":"FLOAT"},{"id":153,"origin_id":-10,"origin_slot":8,"target_id":64,"target_slot":5,"type":"COMBO"},{"id":154,"origin_id":-10,"origin_slot":9,"target_id":64,"target_slot":6,"type":"FLOAT"},{"id":155,"origin_id":-10,"origin_slot":10,"target_id":66,"target_slot":1,"type":"COMBO"},{"id":156,"origin_id":-10,"origin_slot":11,"target_id":52,"target_slot":4,"type":"INT"},{"id":157,"origin_id":-10,"origin_slot":12,"target_id":52,"target_slot":5,"type":"INT"},{"id":158,"origin_id":-10,"origin_slot":13,"target_id":52,"target_slot":6,"type":"FLOAT"},{"id":201,"origin_id":-10,"origin_slot":14,"target_id":54,"target_slot":2,"type":"INT"},{"id":202,"origin_id":-10,"origin_slot":15,"target_id":54,"target_slot":3,"type":"INT"},{"id":203,"origin_id":-10,"origin_slot":16,"target_id":54,"target_slot":4,"type":"COMBO"},{"id":209,"origin_id":-10,"origin_slot":17,"target_id":101,"target_slot":0,"type":"COMBO"},{"id":210,"origin_id":-10,"origin_slot":18,"target_id":102,"target_slot":4,"type":"FLOAT"},{"id":211,"origin_id":-10,"origin_slot":19,"target_id":102,"target_slot":5,"type":"FLOAT"},{"id":212,"origin_id":-10,"origin_slot":20,"target_id":102,"target_slot":6,"type":"FLOAT"},{"id":221,"origin_id":-10,"origin_slot":21,"target_id":51,"target_slot":1,"type":"STRING"},{"id":207,"origin_id":-10,"origin_slot":22,"target_id":102,"target_slot":2,"type":"IMAGE"},{"id":208,"origin_id":-10,"origin_slot":23,"target_id":102,"target_slot":3,"type":"MASK"},{"id":219,"origin_id":-10,"origin_slot":24,"target_id":106,"target_slot":0,"type":"INT"},{"id":220,"origin_id":-10,"origin_slot":25,"target_id":106,"target_slot":1,"type":"INT"}],"extra":{}}]},"config":{},"extra":{"anomalous_hashes":{"14_Anima\\qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"14_Anima/qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"17_Anima\\qwen_image_vae.safetensors":{"hash":"","size":253806246},"17_Anima/qwen_image_vae.safetensors":{"hash":"","size":253806246},"2_Anima\\animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360},"2_Anima/animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360}},"ds":{"scale":0.6588450000000284,"offset":[-696.1676319888263,190.73229081001872]}},"version":0.4} \ No newline at end of file diff --git a/jzl_workflows/data/anima_inpaint_basic.json b/jzl_workflows/data/anima_inpaint_basic.json new file mode 100644 index 0000000..e230715 --- /dev/null +++ b/jzl_workflows/data/anima_inpaint_basic.json @@ -0,0 +1 @@ +{"id":"5dba0a10-43a9-42df-bd53-2c58413ea47d","revision":0,"last_node_id":109,"last_link_id":229,"nodes":[{"id":7,"type":"XB_CanvasLabel","pos":[3400,30],"size":[497,31],"flags":{"allow_interaction":false,"pinned":true},"order":0,"mode":0,"inputs":[],"outputs":[],"title":"更多教程全网搜：机智罗_LX","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":37,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":9,"type":"XB_CanvasLabel","pos":[1180,-260],"size":[2734,298],"flags":{"allow_interaction":false,"pinned":true},"order":1,"mode":0,"inputs":[],"outputs":[],"title":"Anima 局部重绘工作流","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":268,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":67,"type":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","pos":[2480,130],"size":[420,1030],"flags":{"pinned":true},"order":5,"mode":0,"inputs":[{"name":"filename_prefix","type":"STRING","widget":{"name":"filename_prefix"},"link":null},{"name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":null},{"name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":null},{"name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":null},{"name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":null},{"name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":null},{"name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":null},{"name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":null},{"name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":null},{"name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":null},{"label":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":null},{"label":"随机种","name":"seed_1","type":"INT","widget":{"name":"seed_1"},"link":null},{"label":"采样步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":null},{"name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":null},{"label":"分块尺寸","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":null},{"label":"重叠尺寸","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":null},{"label":"解码前执行","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":null},{"name":"name","type":"COMBO","widget":{"name":"name"},"link":null},{"name":"strength","type":"FLOAT","widget":{"name":"strength"},"link":null},{"name":"start_percent","type":"FLOAT","widget":{"name":"start_percent"},"link":null},{"name":"end_percent","type":"FLOAT","widget":{"name":"end_percent"},"link":null},{"label":"提示词","name":"text","type":"STRING","widget":{"name":"text"},"link":null},{"name":"image","type":"IMAGE","link":216},{"name":"mask","shape":7,"type":"MASK","link":217},{"name":"width","type":"INT","widget":{"name":"width"},"link":222},{"name":"height","type":"INT","widget":{"name":"height"},"link":224}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[227]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","previewExposures":[{"name":"$$canvas-image-preview","sourceNodeId":"52","sourcePreviewName":"$$canvas-image-preview"}]},"widgets_values":["图片/Anima/Anima-局部重绘","Anima\\anima-preview3-base.safetensors","Anima\\qwen_3_06b_base.safetensors","Anima\\qwen_image_vae.safetensors","Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1,"关闭",399284861206709,8,1,1920,64,"不做任何清理","Anima\\anima-lllite-inpainting-v2.safetensors",1,0,1,"女孩的眼睛改成红色",1024,1024],"widgets_values_named":{"filename_prefix":"图片/Anima/Anima-局部重绘","unet_name":"Anima\\anima-preview3-base.safetensors","clip_name":"Anima\\qwen_3_06b_base.safetensors","vae_name":"Anima\\qwen_image_vae.safetensors","lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1,"preset":"关闭","seed_1":399284861206709,"steps":8,"cfg":1,"tile_size":1920,"overlap":64,"cleanup":"不做任何清理","name":"Anima\\anima-lllite-inpainting-v2.safetensors","strength":1,"start_percent":0,"end_percent":1,"text":"女孩的眼睛改成红色","width":1024,"height":1024},"color":"#232","bgcolor":"#353"},{"id":104,"type":"LoadImage","pos":[1200,130],"size":[720,710],"flags":{"pinned":true},"order":2,"mode":0,"inputs":[{"localized_name":"图像","name":"image","type":"COMBO","widget":{"name":"image"},"link":null},{"localized_name":"选择文件上传","name":"upload","type":"IMAGEUPLOAD","widget":{"name":"upload"},"link":null}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[215,228]},{"localized_name":"遮罩","name":"MASK","type":"MASK","links":[]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"LoadImage"},"widgets_values":["goat_horned_spirit_girl.png","image"],"widgets_values_named":{"image":"goat_horned_spirit_girl.png","upload":"image"}},{"id":105,"type":"Painter","pos":[1970,130],"size":[460,1030],"flags":{"pinned":true},"order":4,"mode":0,"inputs":[{"localized_name":"图像","name":"image","shape":7,"type":"IMAGE","link":215},{"localized_name":"mask","name":"mask","type":"STRING","widget":{"name":"mask"},"link":null},{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":null},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":null},{"localized_name":"背景颜色","name":"bg_color","type":"COLOR","widget":{"name":"bg_color"},"link":null}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[216]},{"localized_name":"遮罩","name":"MASK","type":"MASK","links":[217]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"Painter","painterTool":"brush","painterBrushSize":69,"painterBrushColor":"#000000","painterBrushOpacity":1,"painterBrushHardness":1},"widgets_values":["painter-105-1793813754736.png [input]",1024,1024,"#000000"],"widgets_values_named":{"mask":"painter-105-1793813754736.png [input]","width":1024,"height":1024,"bg_color":"#000000"}},{"id":107,"type":"XB_ImageParamsMasterMini","pos":[1200,900],"size":[720,260],"flags":{"pinned":true},"order":3,"mode":0,"inputs":[{"localized_name":"画幅比例","name":"aspect_ratio","type":"COMBO","widget":{"name":"aspect_ratio"},"link":null},{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":null},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":null},{"localized_name":"生成数量","name":"batch_size","type":"INT","widget":{"name":"batch_size"},"link":null},{"localized_name":"倍数","name":"multiple","type":"INT","widget":{"name":"multiple"},"link":null}],"outputs":[{"localized_name":"图片宽度","name":"Image Width","type":"INT","links":[222]},{"localized_name":"图片高度","name":"Image Height","type":"INT","links":[224]},{"localized_name":"生成数量","name":"Batch Size","type":"INT","links":null},{"localized_name":"缩放尺寸","name":"Scale Size","type":"INT","links":null}],"properties":{"cnr_id":"XB_ToolBox","ver":"21eb17f086bbb64708a0716e2efeb03fa25bfba2","Node name for S&R":"XB_ImageParamsMasterMini"},"widgets_values":["自定义",1024,1024,1,16],"widgets_values_named":{"aspect_ratio":"自定义","width":1024,"height":1024,"batch_size":1,"multiple":16},"color":"#16727c","bgcolor":"#4F0074"},{"id":108,"type":"ImageCompare","pos":[2950,130],"size":[930,1030],"flags":{"pinned":true},"order":6,"mode":0,"inputs":[{"localized_name":"image_a","name":"image_a","shape":7,"type":"IMAGE","link":227},{"localized_name":"image_b","name":"image_b","shape":7,"type":"IMAGE","link":228},{"localized_name":"compare_view","name":"compare_view","type":"IMAGECOMPARE","widget":{"name":"compare_view"},"link":null}],"outputs":[],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"ImageCompare"},"widgets_values":[],"widgets_values_named":{}}],"links":[[215,104,0,105,0,"IMAGE"],[227,67,0,108,0,"IMAGE"],[228,104,0,108,1,"IMAGE"],[216,105,0,67,22,"IMAGE"],[217,105,1,67,23,"MASK"],[222,107,0,67,24,"INT"],[224,107,1,67,25,"INT"]],"groups":[{"id":1,"title":"Anima-文生图-机智罗","bounding":[1190,60,2700,1110],"color":"#3f789e","flags":{"pinned":true}}],"definitions":{"subgraphs":[{"id":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","version":1,"state":{"lastGroupId":9,"lastNodeId":109,"lastLinkId":229,"lastRerouteId":0},"revision":0,"config":{},"name":"模型加载+采样+解码","inputNode":{"id":-10,"bounding":[240,1540,135.0234375,568]},"outputNode":{"id":-20,"bounding":[2380,1580,128,68]},"inputs":[{"id":"3cd285bf-894e-4810-879d-f17e59ea3450","name":"filename_prefix","type":"STRING","linkIds":[229],"pos":[351.0234375,2064]},{"id":"c4cef3a7-2176-4c21-b780-338df3e1e01b","name":"unet_name","type":"COMBO","linkIds":[146],"pos":[351.0234375,1564]},{"id":"cb8d4371-1883-4f71-865c-84d6da7805b8","name":"clip_name","type":"COMBO","linkIds":[147],"pos":[351.0234375,1584]},{"id":"5e1462bd-1d34-4601-b8d0-810526159ead","name":"vae_name","type":"COMBO","linkIds":[148],"pos":[351.0234375,1604]},{"id":"bcda50ec-08c1-4ca2-8c5a-1a508f31e92f","name":"lora_01","type":"COMBO","linkIds":[149],"pos":[351.0234375,1624]},{"id":"7711dbe8-17f3-4a7c-88f2-dcb8c6087f5b","name":"strength_01","type":"FLOAT","linkIds":[150],"pos":[351.0234375,1644]},{"id":"423da43e-d6e5-441d-9548-aa27f1019e5e","name":"lora_02","type":"COMBO","linkIds":[151],"pos":[351.0234375,1664]},{"id":"ff806aa1-83fe-4271-bfb7-150cde936068","name":"strength_02","type":"FLOAT","linkIds":[152],"pos":[351.0234375,1684]},{"id":"4b26b78f-0a2c-47ac-8793-b37411f94895","name":"lora_03","type":"COMBO","linkIds":[153],"pos":[351.0234375,1704]},{"id":"c2af5451-2693-4f3b-904e-552a90c27946","name":"strength_03","type":"FLOAT","linkIds":[154],"pos":[351.0234375,1724]},{"id":"c68b1c5d-dd69-4b9c-8622-61277812ce2a","name":"preset","type":"COMBO","linkIds":[155],"label":"加速方式","pos":[351.0234375,1744]},{"id":"2f117623-fcc3-4f38-9ecb-e271c51d7f66","name":"seed_1","type":"INT","linkIds":[156],"label":"随机种","pos":[351.0234375,1764]},{"id":"64fbd3d6-b7be-49c3-8a98-3d7348fa1792","name":"steps","type":"INT","linkIds":[157],"label":"采样步数","pos":[351.0234375,1784]},{"id":"19192258-d662-4645-9c3f-4208d6272399","name":"cfg","type":"FLOAT","linkIds":[158],"pos":[351.0234375,1804]},{"id":"8e79a56a-593c-4f95-aa9e-27a51dc2941f","name":"tile_size","type":"INT","linkIds":[201],"label":"分块尺寸","pos":[351.0234375,1824]},{"id":"e24ba591-c57f-43b5-81b7-185ce1f14d21","name":"overlap","type":"INT","linkIds":[202],"label":"重叠尺寸","pos":[351.0234375,1844]},{"id":"4356c50a-2e2c-49ac-8b6d-a26febe29ec0","name":"cleanup","type":"COMBO","linkIds":[203],"label":"解码前执行","pos":[351.0234375,1864]},{"id":"79d52521-67d6-493a-9fb2-f114ca8ae14c","name":"name","type":"COMBO","linkIds":[209],"pos":[351.0234375,1924]},{"id":"90e8ffe4-afe8-4745-bf97-39e1d77de9db","name":"strength","type":"FLOAT","linkIds":[210],"pos":[351.0234375,1944]},{"id":"dd00b1b7-3613-4b82-9a79-2b4c60887e8f","name":"start_percent","type":"FLOAT","linkIds":[211],"pos":[351.0234375,1964]},{"id":"56ebbee1-f7ec-4df5-b950-ff2c7c0d1fe5","name":"end_percent","type":"FLOAT","linkIds":[212],"pos":[351.0234375,1984]},{"id":"9b614adf-fceb-4258-becb-bbaad01a7c59","name":"text","type":"STRING","linkIds":[221],"label":"提示词","pos":[351.0234375,2044]},{"id":"ffd6ddd9-8c05-4dce-8f81-3d2dfcf38f94","name":"image","type":"IMAGE","linkIds":[207],"pos":[351.0234375,1884]},{"id":"df797fd2-96fc-487b-bc50-2421ce883da4","name":"mask","type":"MASK","linkIds":[208],"pos":[351.0234375,1904]},{"id":"309c0c02-9d72-46ea-9f4e-6fe3d6d4533b","name":"width","type":"INT","linkIds":[219],"pos":[351.0234375,2004]},{"id":"3d7c0ea0-9ae1-46d0-8434-da37fcc98ce8","name":"height","type":"INT","linkIds":[220],"pos":[351.0234375,2024]}],"outputs":[{"id":"04f44b9f-158c-4600-bb17-709ca515c335","name":"IMAGE","type":"IMAGE","linkIds":[226],"localized_name":"图像","pos":[2404,1604]}],"widgets":[],"nodes":[{"id":65,"type":"UNETLoader","pos":[510,1190],"size":[440,110],"flags":{},"order":8,"mode":0,"inputs":[{"localized_name":"UNet名称","name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":146},{"localized_name":"数据类型","name":"weight_dtype","type":"COMBO","widget":{"name":"weight_dtype"},"link":null}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[205]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"UNETLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\anima-preview3-base.safetensors","default"],"widgets_values_named":{"unet_name":"Anima\\anima-preview3-base.safetensors","weight_dtype":"default"},"color":"#323","bgcolor":"#535"},{"id":64,"type":"XB_TripleLoraLoaderModelOnly","pos":[510,1350],"size":[440,230],"flags":{},"order":7,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":206},{"localized_name":"LORA-01","name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":149},{"localized_name":"强度-01","name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":150},{"localized_name":"LORA-02","name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":151},{"localized_name":"强度-02","name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":152},{"localized_name":"LORA-03","name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":153},{"localized_name":"强度-03","name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":154}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[133]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_TripleLoraLoaderModelOnly"},"widgets_values":["Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1],"widgets_values_named":{"lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1},"color":"#16727c","bgcolor":"#4F0074"},{"id":51,"type":"CLIPTextEncode","pos":[1080,1180],"size":[340,620],"flags":{},"order":1,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":130},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":221}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","slot_index":0,"links":[102]}],"title":"CLIP Text Encode (Positive Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPTextEncode","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"],"widgets_values_named":{"text":"(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"},"color":"#232","bgcolor":"#353"},{"id":66,"type":"XB_SageAttentionAccelerator","pos":[510,1630],"size":[440,90],"flags":{},"order":9,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":133},{"localized_name":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":155}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[134]}],"properties":{"cnr_id":"XB_ToolBox","ver":"2b291a666b781c0b10117bc51e5edf286ffda5a5","Node name for S&R":"XB_SageAttentionAccelerator","aux_id":"WJLUOXIAO/XB_ToolBox"},"widgets_values":["comfy kitchen attention"],"widgets_values_named":{"preset":"comfy kitchen attention"},"color":"#16727c","bgcolor":"#4F0074"},{"id":63,"type":"CLIPLoader","pos":[510,1770],"size":[440,140],"flags":{},"order":6,"mode":0,"inputs":[{"localized_name":"CLIP名称","name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":147},{"localized_name":"类型","name":"type","type":"COMBO","widget":{"name":"type"},"link":null},{"localized_name":"设备","name":"device","shape":7,"type":"COMBO","widget":{"name":"device"},"link":null}],"outputs":[{"localized_name":"CLIP","name":"CLIP","type":"CLIP","links":[130,131]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_3_06b_base.safetensors","stable_diffusion","default"],"widgets_values_named":{"clip_name":"Anima\\qwen_3_06b_base.safetensors","type":"stable_diffusion","device":"default"},"color":"#323","bgcolor":"#535"},{"id":50,"type":"CLIPTextEncode","pos":[1080,1850],"size":[340,130],"flags":{},"order":0,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":131},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":null}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","links":[103]}],"title":"CLIP Text Encode (Negative Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.3.65","Node name for S&R":"CLIPTextEncode","enableTabs":false,"tabWidth":65,"tabXOffset":10,"hasSecondTab":false,"secondTabText":"Send Back","secondTabOffset":80,"secondTabWidth":65},"widgets_values":["worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"],"widgets_values_named":{"text":"worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"},"color":"#322","bgcolor":"#533"},{"id":62,"type":"VAELoader","pos":[510,1960],"size":[440,90],"flags":{},"order":5,"mode":0,"inputs":[{"localized_name":"vae名称","name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":148}],"outputs":[{"localized_name":"VAE","name":"VAE","type":"VAE","links":[132]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"VAELoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_image_vae.safetensors"],"widgets_values_named":{"vae_name":"Anima\\qwen_image_vae.safetensors"},"color":"#323","bgcolor":"#535"},{"id":52,"type":"KSampler","pos":[1480,1460],"size":[310,410],"flags":{},"order":2,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":134},{"localized_name":"正面条件","name":"positive","type":"CONDITIONING","link":102},{"localized_name":"负面条件","name":"negative","type":"CONDITIONING","link":103},{"localized_name":"Latent图像","name":"latent_image","type":"LATENT","link":218},{"localized_name":"种子","name":"seed","type":"INT","widget":{"name":"seed"},"link":156},{"localized_name":"步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":157},{"localized_name":"cfg","name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":158},{"localized_name":"采样器名称","name":"sampler_name","type":"COMBO","widget":{"name":"sampler_name"},"link":null},{"localized_name":"调度器","name":"scheduler","type":"COMBO","widget":{"name":"scheduler"},"link":null},{"localized_name":"降噪","name":"denoise","type":"FLOAT","widget":{"name":"denoise"},"link":null}],"outputs":[{"localized_name":"Latent","name":"LATENT","type":"LATENT","links":[105]}],"properties":{"cnr_id":"comfy-core","ver":"0.42.1","Node name for S&R":"KSampler"},"widgets_values":[466366138910810,"randomize",8,1,"res_multistep","simple",1],"widgets_values_named":{"seed":466366138910810,"control_after_generate":"randomize","steps":8,"cfg":1,"sampler_name":"res_multistep","scheduler":"simple","denoise":1}},{"id":53,"type":"XB_PlaySound","pos":[1520,1400],"size":[270,110],"flags":{"collapsed":true},"order":3,"mode":0,"inputs":[{"localized_name":"任意输入","name":"any","type":"*","link":104},{"localized_name":"播放模式","name":"mode","type":"COMBO","widget":{"name":"mode"},"link":null},{"localized_name":"音量","name":"volume","type":"FLOAT","widget":{"name":"volume"},"link":null},{"localized_name":"声音文件","name":"file","type":"STRING","widget":{"name":"file"},"link":null}],"outputs":[{"localized_name":"任意输出","name":"any","shape":6,"type":"*","links":[]}],"title":"🔊 播放声音","properties":{"cnr_id":"XB_ToolBox","ver":"7058d1613c09977817f62bf226065ff90d55cb61","Node name for S&R":"XB_PlaySound"},"widgets_values":["always",0.5,"notify.mp3"],"widgets_values_named":{"mode":"always","volume":0.5,"file":"notify.mp3"},"color":"#16727c","bgcolor":"#4F0074"},{"id":54,"type":"XB_VAEDecodeTiledImage","pos":[1480,1180],"size":[310,170],"flags":{},"order":4,"mode":0,"inputs":[{"localized_name":"潜空间","name":"samples","type":"LATENT","link":105},{"localized_name":"VAE","name":"vae","type":"VAE","link":132},{"localized_name":"分块大小","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":201},{"localized_name":"重叠","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":202},{"localized_name":"执行前清理","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":203}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[104,225]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_VAEDecodeTiledImage"},"widgets_values":[1920,64,"不做任何清理"],"widgets_values_named":{"tile_size":1920,"overlap":64,"cleanup":"不做任何清理"},"color":"#16727c","bgcolor":"#4F0074"},{"id":101,"type":"ModelPatchLoader","pos":[510,2100],"size":[440,90],"flags":{},"order":10,"mode":0,"inputs":[{"localized_name":"名称","name":"name","type":"COMBO","widget":{"name":"name"},"link":209}],"outputs":[{"localized_name":"模型补丁","name":"MODEL_PATCH","type":"MODEL_PATCH","links":[204]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"ModelPatchLoader","models":[{"name":"anima-lllite-inpainting-v2.safetensors","url":"https://huggingface.co/Comfy-Org/Anima-LLLite/resolve/main/model_patches/anima-lllite-inpainting-v2.safetensors","directory":"model_patches"}]},"widgets_values":["Anima\\anima-lllite-inpainting-v2.safetensors"],"widgets_values_named":{"name":"Anima\\anima-lllite-inpainting-v2.safetensors"}},{"id":102,"type":"AnimaLLLiteApply","pos":[1000,2090],"size":[270,240],"flags":{},"order":11,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":205},{"localized_name":"模型补丁","name":"model_patch","type":"MODEL_PATCH","link":204},{"localized_name":"图像","name":"image","type":"IMAGE","link":207},{"localized_name":"掩码","name":"mask","shape":7,"type":"MASK","link":208},{"localized_name":"强度","name":"strength","type":"FLOAT","widget":{"name":"strength"},"link":210},{"localized_name":"起始百分比","name":"start_percent","type":"FLOAT","widget":{"name":"start_percent"},"link":211},{"localized_name":"结束百分比","name":"end_percent","type":"FLOAT","widget":{"name":"end_percent"},"link":212}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[206]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","Node name for S&R":"AnimaLLLiteApply"},"widgets_values":[1,0,1],"widgets_values_named":{"strength":1,"start_percent":0,"end_percent":1}},{"id":106,"type":"EmptyLatentImage","pos":[550,2260],"size":[310,150],"flags":{},"order":12,"mode":0,"inputs":[{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":219},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":220},{"localized_name":"批量大小","name":"batch_size","type":"INT","widget":{"name":"batch_size"},"link":null}],"outputs":[{"localized_name":"Latent","name":"LATENT","type":"LATENT","links":[218]}],"properties":{"cnr_id":"comfy-core","ver":"0.3.40","Node name for S&R":"EmptyLatentImage","enableTabs":false,"tabWidth":65,"tabXOffset":10,"hasSecondTab":false,"secondTabText":"Send Back","secondTabOffset":80,"secondTabWidth":65},"widgets_values":[1024,1024,1],"widgets_values_named":{"width":1024,"height":1024,"batch_size":1}},{"id":109,"type":"SaveImage","pos":[1850,1200],"size":[490,620],"flags":{},"order":13,"mode":0,"inputs":[{"localized_name":"图片","name":"images","type":"IMAGE","link":225},{"localized_name":"文件名前缀","name":"filename_prefix","type":"STRING","widget":{"name":"filename_prefix"},"link":229}],"outputs":[{"localized_name":"images","name":"images","type":"IMAGE","links":[226]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"SaveImage","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["图片/Anima/文生图/%date:yyyy-MM-dd%/Anima-文生图"],"widgets_values_named":{"filename_prefix":"图片/Anima/文生图/%date:yyyy-MM-dd%/Anima-文生图"},"color":"#223","bgcolor":"#335"}],"groups":[],"links":[{"id":130,"origin_id":63,"origin_slot":0,"target_id":51,"target_slot":0,"type":"CLIP"},{"id":133,"origin_id":64,"origin_slot":0,"target_id":66,"target_slot":0,"type":"MODEL"},{"id":131,"origin_id":63,"origin_slot":0,"target_id":50,"target_slot":0,"type":"CLIP"},{"id":134,"origin_id":66,"origin_slot":0,"target_id":52,"target_slot":0,"type":"MODEL"},{"id":102,"origin_id":51,"origin_slot":0,"target_id":52,"target_slot":1,"type":"CONDITIONING"},{"id":103,"origin_id":50,"origin_slot":0,"target_id":52,"target_slot":2,"type":"CONDITIONING"},{"id":104,"origin_id":54,"origin_slot":0,"target_id":53,"target_slot":0,"type":"IMAGE"},{"id":105,"origin_id":52,"origin_slot":0,"target_id":54,"target_slot":0,"type":"LATENT"},{"id":132,"origin_id":62,"origin_slot":0,"target_id":54,"target_slot":1,"type":"VAE"},{"id":204,"origin_id":101,"origin_slot":0,"target_id":102,"target_slot":1,"type":"MODEL_PATCH"},{"id":205,"origin_id":65,"origin_slot":0,"target_id":102,"target_slot":0,"type":"MODEL"},{"id":206,"origin_id":102,"origin_slot":0,"target_id":64,"target_slot":0,"type":"MODEL"},{"id":218,"origin_id":106,"origin_slot":0,"target_id":52,"target_slot":3,"type":"LATENT"},{"id":225,"origin_id":54,"origin_slot":0,"target_id":109,"target_slot":0,"type":"IMAGE"},{"id":226,"origin_id":109,"origin_slot":0,"target_id":-20,"target_slot":0,"type":"IMAGE"},{"id":229,"origin_id":-10,"origin_slot":0,"target_id":109,"target_slot":1,"type":"STRING"},{"id":146,"origin_id":-10,"origin_slot":1,"target_id":65,"target_slot":0,"type":"COMBO"},{"id":147,"origin_id":-10,"origin_slot":2,"target_id":63,"target_slot":0,"type":"COMBO"},{"id":148,"origin_id":-10,"origin_slot":3,"target_id":62,"target_slot":0,"type":"COMBO"},{"id":149,"origin_id":-10,"origin_slot":4,"target_id":64,"target_slot":1,"type":"COMBO"},{"id":150,"origin_id":-10,"origin_slot":5,"target_id":64,"target_slot":2,"type":"FLOAT"},{"id":151,"origin_id":-10,"origin_slot":6,"target_id":64,"target_slot":3,"type":"COMBO"},{"id":152,"origin_id":-10,"origin_slot":7,"target_id":64,"target_slot":4,"type":"FLOAT"},{"id":153,"origin_id":-10,"origin_slot":8,"target_id":64,"target_slot":5,"type":"COMBO"},{"id":154,"origin_id":-10,"origin_slot":9,"target_id":64,"target_slot":6,"type":"FLOAT"},{"id":155,"origin_id":-10,"origin_slot":10,"target_id":66,"target_slot":1,"type":"COMBO"},{"id":156,"origin_id":-10,"origin_slot":11,"target_id":52,"target_slot":4,"type":"INT"},{"id":157,"origin_id":-10,"origin_slot":12,"target_id":52,"target_slot":5,"type":"INT"},{"id":158,"origin_id":-10,"origin_slot":13,"target_id":52,"target_slot":6,"type":"FLOAT"},{"id":201,"origin_id":-10,"origin_slot":14,"target_id":54,"target_slot":2,"type":"INT"},{"id":202,"origin_id":-10,"origin_slot":15,"target_id":54,"target_slot":3,"type":"INT"},{"id":203,"origin_id":-10,"origin_slot":16,"target_id":54,"target_slot":4,"type":"COMBO"},{"id":209,"origin_id":-10,"origin_slot":17,"target_id":101,"target_slot":0,"type":"COMBO"},{"id":210,"origin_id":-10,"origin_slot":18,"target_id":102,"target_slot":4,"type":"FLOAT"},{"id":211,"origin_id":-10,"origin_slot":19,"target_id":102,"target_slot":5,"type":"FLOAT"},{"id":212,"origin_id":-10,"origin_slot":20,"target_id":102,"target_slot":6,"type":"FLOAT"},{"id":221,"origin_id":-10,"origin_slot":21,"target_id":51,"target_slot":1,"type":"STRING"},{"id":207,"origin_id":-10,"origin_slot":22,"target_id":102,"target_slot":2,"type":"IMAGE"},{"id":208,"origin_id":-10,"origin_slot":23,"target_id":102,"target_slot":3,"type":"MASK"},{"id":219,"origin_id":-10,"origin_slot":24,"target_id":106,"target_slot":0,"type":"INT"},{"id":220,"origin_id":-10,"origin_slot":25,"target_id":106,"target_slot":1,"type":"INT"}],"extra":{}}]},"config":{},"extra":{"anomalous_hashes":{"14_Anima\\qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"14_Anima/qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"17_Anima\\qwen_image_vae.safetensors":{"hash":"","size":253806246},"17_Anima/qwen_image_vae.safetensors":{"hash":"","size":253806246},"2_Anima\\animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360},"2_Anima/animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360}},"ds":{"scale":0.4500000000000192,"offset":[98.35376826935745,1046.2968408582683]}},"version":0.4} \ No newline at end of file diff --git a/jzl_workflows/data/anima_t2i_basic.json b/jzl_workflows/data/anima_t2i_basic.json new file mode 100644 index 0000000..97b5fb4 --- /dev/null +++ b/jzl_workflows/data/anima_t2i_basic.json @@ -0,0 +1 @@ +{"id":"3f934f13-847a-438e-89b5-86edabd9ea87","revision":0,"last_node_id":100,"last_link_id":203,"nodes":[{"id":7,"type":"XB_CanvasLabel","pos":[2620,30],"size":[497,31],"flags":{"allow_interaction":false,"pinned":true},"order":0,"mode":0,"inputs":[],"outputs":[],"title":"更多教程全网搜：机智罗_LX","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":37,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":9,"type":"XB_CanvasLabel","pos":[1180,-200],"size":[1947,233],"flags":{"allow_interaction":false,"pinned":true},"order":1,"mode":0,"inputs":[],"outputs":[],"title":"Anima 文生图工作流","properties":{"cnr_id":"XB_ToolBox","ver":"75a7af8c69a8f7e1b98e265539bd0bb0a56fd95d","Node name for S&R":"XB_CanvasLabel","fontSize":211,"fontColor":"#ffffff","textAlign":"left","backgroundColor":"transparent","padding":0,"borderRadius":0,"angle":0,"aux_id":"wjluoxiao/XB_ToolBox"},"widgets_values":[""],"widgets_values_named":{"xb_label_view":""},"color":"transparent","bgcolor":"transparent"},{"id":20,"type":"SaveImage","pos":[2300,130],"size":[800,830],"flags":{"pinned":true},"order":4,"mode":0,"inputs":[{"localized_name":"图片","name":"images","type":"IMAGE","link":137},{"localized_name":"文件名前缀","name":"filename_prefix","type":"STRING","widget":{"name":"filename_prefix"},"link":null}],"outputs":[{"localized_name":"images","name":"images","type":"IMAGE","links":null}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"SaveImage","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["图片/Anima/Anima-文生图"],"widgets_values_named":{"filename_prefix":"图片/Anima/Anima-文生图"},"color":"#223","bgcolor":"#335"},{"id":45,"type":"XB_ImagePromptPreset","pos":[1200,130],"size":[600,810],"flags":{"pinned":true},"order":2,"mode":0,"inputs":[{"localized_name":"空latent类型","name":"latent_kind","type":"COMBO","widget":{"name":"latent_kind"},"link":null},{"localized_name":"输出语言","name":"output_lang","type":"COMBO","widget":{"name":"output_lang"},"link":null},{"localized_name":"预设模式","name":"preset_mode","type":"COMBO","widget":{"name":"preset_mode"},"link":null},{"localized_name":"预设句（右上角按钮管理预设）","name":"three_view_text","type":"STRING","widget":{"name":"three_view_text"},"link":null},{"localized_name":"画幅比例","name":"aspect_ratio","type":"COMBO","widget":{"name":"aspect_ratio"},"link":null},{"localized_name":"宽度","name":"width","type":"INT","widget":{"name":"width"},"link":null},{"localized_name":"高度","name":"height","type":"INT","widget":{"name":"height"},"link":null},{"localized_name":"生成数量","name":"batch_size","type":"INT","widget":{"name":"batch_size"},"link":null}],"outputs":[{"localized_name":"提示词","name":"提示词","type":"STRING","links":[144]},{"localized_name":"空latent","name":"空latent","type":"LATENT","links":[145]}],"properties":{"cnr_id":"XB_ToolBox","ver":"21eb17f086bbb64708a0716e2efeb03fa25bfba2","Node name for S&R":"XB_ImagePromptPreset"},"widgets_values":["Anima","中文 [ZH]","无预设","","Free",1024,1024,1,"(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile","",""],"widgets_values_named":{"latent_kind":"Anima","output_lang":"中文 [ZH]","preset_mode":"无预设","three_view_text":"","aspect_ratio":"Free","width":1024,"height":1024,"batch_size":1,"internal_prompt":"(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile","manager_settings":"","xb_image_preset":""},"color":"#16727c","bgcolor":"#4F0074"},{"id":67,"type":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","pos":[1850,130],"size":[400,830],"flags":{"pinned":true},"order":3,"mode":0,"inputs":[{"label":"提示词","name":"text","type":"STRING","link":144},{"label":"Latent","name":"latent_image_1","type":"LATENT","link":145},{"name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":null},{"name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":null},{"name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":null},{"name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":null},{"name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":null},{"name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":null},{"name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":null},{"name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":null},{"name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":null},{"label":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":null},{"label":"随机种","name":"seed_1","type":"INT","widget":{"name":"seed_1"},"link":null},{"label":"采样步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":null},{"name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":null},{"label":"分块尺寸","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":null},{"label":"重叠尺寸","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":null},{"label":"解码前执行","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":null}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[137]}],"properties":{"cnr_id":"comfy-core","ver":"0.43.2","previewExposures":[{"name":"$$canvas-image-preview","sourceNodeId":"52","sourcePreviewName":"$$canvas-image-preview"}]},"widgets_values":["Anima\\anima-preview3-base.safetensors","Anima\\qwen_3_06b_base.safetensors","Anima\\qwen_image_vae.safetensors","Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1,"关闭",894692508254696,8,1,1920,64,"不做任何清理"],"widgets_values_named":{"unet_name":"Anima\\anima-preview3-base.safetensors","clip_name":"Anima\\qwen_3_06b_base.safetensors","vae_name":"Anima\\qwen_image_vae.safetensors","lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1,"preset":"关闭","seed_1":894692508254696,"steps":8,"cfg":1,"tile_size":1920,"overlap":64,"cleanup":"不做任何清理"},"color":"#232","bgcolor":"#353"}],"links":[[137,67,0,20,0,"IMAGE"],[144,45,0,67,0,"STRING"],[145,45,1,67,1,"LATENT"]],"groups":[{"id":1,"title":"Anima-文生图-机智罗","bounding":[1190,60,1920,910],"color":"#3f789e","flags":{"pinned":true}}],"definitions":{"subgraphs":[{"id":"0f96aaef-b1d7-42ed-9631-ee228d65f2cb","version":1,"state":{"lastGroupId":9,"lastNodeId":100,"lastLinkId":203,"lastRerouteId":0},"revision":0,"config":{},"name":"模型加载+采样+解码","inputNode":{"id":-10,"bounding":[240,1540,128,408]},"outputNode":{"id":-20,"bounding":[1880,1590,128,68]},"inputs":[{"id":"83e7d6bc-b591-4878-871e-e42067cbce29","name":"text","type":"STRING","linkIds":[140],"label":"提示词","pos":[344,1564]},{"id":"f0612cf0-c08b-48bf-a3bf-ccb98bc77971","name":"latent_image_1","type":"LATENT","linkIds":[142],"label":"Latent","pos":[344,1584]},{"id":"c4cef3a7-2176-4c21-b780-338df3e1e01b","name":"unet_name","type":"COMBO","linkIds":[146],"pos":[344,1604]},{"id":"cb8d4371-1883-4f71-865c-84d6da7805b8","name":"clip_name","type":"COMBO","linkIds":[147],"pos":[344,1624]},{"id":"5e1462bd-1d34-4601-b8d0-810526159ead","name":"vae_name","type":"COMBO","linkIds":[148],"pos":[344,1644]},{"id":"bcda50ec-08c1-4ca2-8c5a-1a508f31e92f","name":"lora_01","type":"COMBO","linkIds":[149],"pos":[344,1664]},{"id":"7711dbe8-17f3-4a7c-88f2-dcb8c6087f5b","name":"strength_01","type":"FLOAT","linkIds":[150],"pos":[344,1684]},{"id":"423da43e-d6e5-441d-9548-aa27f1019e5e","name":"lora_02","type":"COMBO","linkIds":[151],"pos":[344,1704]},{"id":"ff806aa1-83fe-4271-bfb7-150cde936068","name":"strength_02","type":"FLOAT","linkIds":[152],"pos":[344,1724]},{"id":"4b26b78f-0a2c-47ac-8793-b37411f94895","name":"lora_03","type":"COMBO","linkIds":[153],"pos":[344,1744]},{"id":"c2af5451-2693-4f3b-904e-552a90c27946","name":"strength_03","type":"FLOAT","linkIds":[154],"pos":[344,1764]},{"id":"c68b1c5d-dd69-4b9c-8622-61277812ce2a","name":"preset","type":"COMBO","linkIds":[155],"label":"加速方式","pos":[344,1784]},{"id":"2f117623-fcc3-4f38-9ecb-e271c51d7f66","name":"seed_1","type":"INT","linkIds":[156],"label":"随机种","pos":[344,1804]},{"id":"64fbd3d6-b7be-49c3-8a98-3d7348fa1792","name":"steps","type":"INT","linkIds":[157],"label":"采样步数","pos":[344,1824]},{"id":"19192258-d662-4645-9c3f-4208d6272399","name":"cfg","type":"FLOAT","linkIds":[158],"pos":[344,1844]},{"id":"8e79a56a-593c-4f95-aa9e-27a51dc2941f","name":"tile_size","type":"INT","linkIds":[201],"label":"分块尺寸","pos":[344,1864]},{"id":"e24ba591-c57f-43b5-81b7-185ce1f14d21","name":"overlap","type":"INT","linkIds":[202],"label":"重叠尺寸","pos":[344,1884]},{"id":"4356c50a-2e2c-49ac-8b6d-a26febe29ec0","name":"cleanup","type":"COMBO","linkIds":[203],"label":"解码前执行","pos":[344,1904]}],"outputs":[{"id":"04f44b9f-158c-4600-bb17-709ca515c335","name":"IMAGE","type":"IMAGE","linkIds":[106],"localized_name":"图像","pos":[1904,1614]}],"widgets":[],"nodes":[{"id":65,"type":"UNETLoader","pos":[510,1190],"size":[440,110],"flags":{},"order":8,"mode":0,"inputs":[{"localized_name":"UNet名称","name":"unet_name","type":"COMBO","widget":{"name":"unet_name"},"link":146},{"localized_name":"数据类型","name":"weight_dtype","type":"COMBO","widget":{"name":"weight_dtype"},"link":null}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[129]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"UNETLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\anima-preview3-base.safetensors","default"],"widgets_values_named":{"unet_name":"Anima\\anima-preview3-base.safetensors","weight_dtype":"default"},"color":"#323","bgcolor":"#535"},{"id":64,"type":"XB_TripleLoraLoaderModelOnly","pos":[510,1350],"size":[440,230],"flags":{},"order":7,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":129},{"localized_name":"LORA-01","name":"lora_01","type":"COMBO","widget":{"name":"lora_01"},"link":149},{"localized_name":"强度-01","name":"strength_01","type":"FLOAT","widget":{"name":"strength_01"},"link":150},{"localized_name":"LORA-02","name":"lora_02","type":"COMBO","widget":{"name":"lora_02"},"link":151},{"localized_name":"强度-02","name":"strength_02","type":"FLOAT","widget":{"name":"strength_02"},"link":152},{"localized_name":"LORA-03","name":"lora_03","type":"COMBO","widget":{"name":"lora_03"},"link":153},{"localized_name":"强度-03","name":"strength_03","type":"FLOAT","widget":{"name":"strength_03"},"link":154}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[133]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_TripleLoraLoaderModelOnly"},"widgets_values":["Anima\\anima-turbo-lora-v0.2.safetensors",1,"关闭",1,"关闭",1],"widgets_values_named":{"lora_01":"Anima\\anima-turbo-lora-v0.2.safetensors","strength_01":1,"lora_02":"关闭","strength_02":1,"lora_03":"关闭","strength_03":1},"color":"#16727c","bgcolor":"#4F0074"},{"id":51,"type":"CLIPTextEncode","pos":[1080,1180],"size":[340,620],"flags":{},"order":1,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":130},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":141}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","slot_index":0,"links":[102]}],"title":"CLIP Text Encode (Positive Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPTextEncode","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"],"widgets_values_named":{"text":"(masterpiece, best quality, very aesthetic, ultra detailed), intricate details, 4k, aausagi, long hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile"},"color":"#232","bgcolor":"#353"},{"id":66,"type":"XB_SageAttentionAccelerator","pos":[510,1630],"size":[440,90],"flags":{},"order":9,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":133},{"localized_name":"加速方式","name":"preset","type":"COMBO","widget":{"name":"preset"},"link":155}],"outputs":[{"localized_name":"模型","name":"MODEL","type":"MODEL","links":[134]}],"properties":{"cnr_id":"XB_ToolBox","ver":"2b291a666b781c0b10117bc51e5edf286ffda5a5","Node name for S&R":"XB_SageAttentionAccelerator","aux_id":"WJLUOXIAO/XB_ToolBox"},"widgets_values":["comfy kitchen attention"],"widgets_values_named":{"preset":"comfy kitchen attention"},"color":"#16727c","bgcolor":"#4F0074"},{"id":63,"type":"CLIPLoader","pos":[510,1770],"size":[440,140],"flags":{},"order":6,"mode":0,"inputs":[{"localized_name":"CLIP名称","name":"clip_name","type":"COMBO","widget":{"name":"clip_name"},"link":147},{"localized_name":"类型","name":"type","type":"COMBO","widget":{"name":"type"},"link":null},{"localized_name":"设备","name":"device","shape":7,"type":"COMBO","widget":{"name":"device"},"link":null}],"outputs":[{"localized_name":"CLIP","name":"CLIP","type":"CLIP","links":[130,131]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"CLIPLoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_3_06b_base.safetensors","stable_diffusion","default"],"widgets_values_named":{"clip_name":"Anima\\qwen_3_06b_base.safetensors","type":"stable_diffusion","device":"default"},"color":"#323","bgcolor":"#535"},{"id":50,"type":"CLIPTextEncode","pos":[1080,1850],"size":[340,130],"flags":{},"order":0,"mode":0,"inputs":[{"localized_name":"clip","name":"clip","type":"CLIP","link":131},{"localized_name":"文本","name":"text","type":"STRING","widget":{"name":"text"},"link":null}],"outputs":[{"localized_name":"条件","name":"CONDITIONING","type":"CONDITIONING","links":[103]}],"title":"CLIP Text Encode (Negative Prompt)","properties":{"cnr_id":"comfy-core","ver":"0.3.65","Node name for S&R":"CLIPTextEncode","enableTabs":false,"tabWidth":65,"tabXOffset":10,"hasSecondTab":false,"secondTabText":"Send Back","secondTabOffset":80,"secondTabWidth":65},"widgets_values":["worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"],"widgets_values_named":{"text":"worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia"},"color":"#322","bgcolor":"#533"},{"id":62,"type":"VAELoader","pos":[510,1960],"size":[440,90],"flags":{},"order":5,"mode":0,"inputs":[{"localized_name":"vae名称","name":"vae_name","type":"COMBO","widget":{"name":"vae_name"},"link":148}],"outputs":[{"localized_name":"VAE","name":"VAE","type":"VAE","links":[132]}],"properties":{"cnr_id":"comfy-core","ver":"0.16.3","Node name for S&R":"VAELoader","ue_properties":{"widget_ue_connectable":{},"version":"7.7","input_ue_unconnectable":{}}},"widgets_values":["Anima\\qwen_image_vae.safetensors"],"widgets_values_named":{"vae_name":"Anima\\qwen_image_vae.safetensors"},"color":"#323","bgcolor":"#535"},{"id":52,"type":"KSampler","pos":[1480,1460],"size":[310,410],"flags":{},"order":2,"mode":0,"inputs":[{"localized_name":"模型","name":"model","type":"MODEL","link":134},{"localized_name":"正面条件","name":"positive","type":"CONDITIONING","link":102},{"localized_name":"负面条件","name":"negative","type":"CONDITIONING","link":103},{"localized_name":"Latent图像","name":"latent_image","type":"LATENT","link":142},{"localized_name":"种子","name":"seed","type":"INT","widget":{"name":"seed"},"link":156},{"localized_name":"步数","name":"steps","type":"INT","widget":{"name":"steps"},"link":157},{"localized_name":"cfg","name":"cfg","type":"FLOAT","widget":{"name":"cfg"},"link":158},{"localized_name":"采样器名称","name":"sampler_name","type":"COMBO","widget":{"name":"sampler_name"},"link":null},{"localized_name":"调度器","name":"scheduler","type":"COMBO","widget":{"name":"scheduler"},"link":null},{"localized_name":"降噪","name":"denoise","type":"FLOAT","widget":{"name":"denoise"},"link":null}],"outputs":[{"localized_name":"Latent","name":"LATENT","type":"LATENT","links":[105]}],"properties":{"cnr_id":"comfy-core","ver":"0.42.1","Node name for S&R":"KSampler"},"widgets_values":[466366138910810,"randomize",8,1,"res_multistep","simple",1],"widgets_values_named":{"seed":466366138910810,"control_after_generate":"randomize","steps":8,"cfg":1,"sampler_name":"res_multistep","scheduler":"simple","denoise":1}},{"id":53,"type":"XB_PlaySound","pos":[1520,1400],"size":[270,110],"flags":{"collapsed":true},"order":3,"mode":0,"inputs":[{"localized_name":"任意输入","name":"any","type":"*","link":104},{"localized_name":"播放模式","name":"mode","type":"COMBO","widget":{"name":"mode"},"link":null},{"localized_name":"音量","name":"volume","type":"FLOAT","widget":{"name":"volume"},"link":null},{"localized_name":"声音文件","name":"file","type":"STRING","widget":{"name":"file"},"link":null}],"outputs":[{"localized_name":"任意输出","name":"any","shape":6,"type":"*","links":[]}],"title":"🔊 播放声音","properties":{"cnr_id":"XB_ToolBox","ver":"7058d1613c09977817f62bf226065ff90d55cb61","Node name for S&R":"XB_PlaySound"},"widgets_values":["always",0.5,"notify.mp3"],"widgets_values_named":{"mode":"always","volume":0.5,"file":"notify.mp3"},"color":"#16727c","bgcolor":"#4F0074"},{"id":54,"type":"XB_VAEDecodeTiledImage","pos":[1480,1180],"size":[310,170],"flags":{},"order":4,"mode":0,"inputs":[{"localized_name":"潜空间","name":"samples","type":"LATENT","link":105},{"localized_name":"VAE","name":"vae","type":"VAE","link":132},{"localized_name":"分块大小","name":"tile_size","type":"INT","widget":{"name":"tile_size"},"link":201},{"localized_name":"重叠","name":"overlap","type":"INT","widget":{"name":"overlap"},"link":202},{"localized_name":"执行前清理","name":"cleanup","type":"COMBO","widget":{"name":"cleanup"},"link":203}],"outputs":[{"localized_name":"图像","name":"IMAGE","type":"IMAGE","links":[104,106]}],"properties":{"cnr_id":"XB_ToolBox","ver":"35a6c6388058434d0b3a5cb64b39677726a729d5","Node name for S&R":"XB_VAEDecodeTiledImage"},"widgets_values":[1920,64,"不做任何清理"],"widgets_values_named":{"tile_size":1920,"overlap":64,"cleanup":"不做任何清理"},"color":"#16727c","bgcolor":"#4F0074"},{"id":68,"type":"Reroute","pos":[970,1520],"size":[80,30],"flags":{},"order":10,"mode":0,"inputs":[{"name":"","type":"*","link":140}],"outputs":[{"name":"","type":"*","links":[141]}],"properties":{"showOutputText":false,"horizontal":false,"ue_properties":{"version":"7.7","widget_ue_connectable":{},"input_ue_unconnectable":{}}},"color":"#222","bgcolor":"#000"}],"groups":[],"links":[{"id":129,"origin_id":65,"origin_slot":0,"target_id":64,"target_slot":0,"type":"MODEL"},{"id":130,"origin_id":63,"origin_slot":0,"target_id":51,"target_slot":0,"type":"CLIP"},{"id":133,"origin_id":64,"origin_slot":0,"target_id":66,"target_slot":0,"type":"MODEL"},{"id":131,"origin_id":63,"origin_slot":0,"target_id":50,"target_slot":0,"type":"CLIP"},{"id":134,"origin_id":66,"origin_slot":0,"target_id":52,"target_slot":0,"type":"MODEL"},{"id":102,"origin_id":51,"origin_slot":0,"target_id":52,"target_slot":1,"type":"CONDITIONING"},{"id":103,"origin_id":50,"origin_slot":0,"target_id":52,"target_slot":2,"type":"CONDITIONING"},{"id":104,"origin_id":54,"origin_slot":0,"target_id":53,"target_slot":0,"type":"IMAGE"},{"id":105,"origin_id":52,"origin_slot":0,"target_id":54,"target_slot":0,"type":"LATENT"},{"id":132,"origin_id":62,"origin_slot":0,"target_id":54,"target_slot":1,"type":"VAE"},{"id":106,"origin_id":54,"origin_slot":0,"target_id":-20,"target_slot":0,"type":"IMAGE"},{"id":141,"origin_id":68,"origin_slot":0,"target_id":51,"target_slot":1,"type":"STRING"},{"id":140,"origin_id":-10,"origin_slot":0,"target_id":68,"target_slot":0,"type":"*"},{"id":142,"origin_id":-10,"origin_slot":1,"target_id":52,"target_slot":3,"type":"LATENT"},{"id":146,"origin_id":-10,"origin_slot":2,"target_id":65,"target_slot":0,"type":"COMBO"},{"id":147,"origin_id":-10,"origin_slot":3,"target_id":63,"target_slot":0,"type":"COMBO"},{"id":148,"origin_id":-10,"origin_slot":4,"target_id":62,"target_slot":0,"type":"COMBO"},{"id":149,"origin_id":-10,"origin_slot":5,"target_id":64,"target_slot":1,"type":"COMBO"},{"id":150,"origin_id":-10,"origin_slot":6,"target_id":64,"target_slot":2,"type":"FLOAT"},{"id":151,"origin_id":-10,"origin_slot":7,"target_id":64,"target_slot":3,"type":"COMBO"},{"id":152,"origin_id":-10,"origin_slot":8,"target_id":64,"target_slot":4,"type":"FLOAT"},{"id":153,"origin_id":-10,"origin_slot":9,"target_id":64,"target_slot":5,"type":"COMBO"},{"id":154,"origin_id":-10,"origin_slot":10,"target_id":64,"target_slot":6,"type":"FLOAT"},{"id":155,"origin_id":-10,"origin_slot":11,"target_id":66,"target_slot":1,"type":"COMBO"},{"id":156,"origin_id":-10,"origin_slot":12,"target_id":52,"target_slot":4,"type":"INT"},{"id":157,"origin_id":-10,"origin_slot":13,"target_id":52,"target_slot":5,"type":"INT"},{"id":158,"origin_id":-10,"origin_slot":14,"target_id":52,"target_slot":6,"type":"FLOAT"},{"id":201,"origin_id":-10,"origin_slot":15,"target_id":54,"target_slot":2,"type":"INT"},{"id":202,"origin_id":-10,"origin_slot":16,"target_id":54,"target_slot":3,"type":"INT"},{"id":203,"origin_id":-10,"origin_slot":17,"target_id":54,"target_slot":4,"type":"COMBO"}],"extra":{}}]},"config":{},"extra":{"anomalous_hashes":{"14_Anima\\qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"14_Anima/qwen_3_06b_base.safetensors":{"hash":"","size":1192135096},"17_Anima\\qwen_image_vae.safetensors":{"hash":"","size":253806246},"17_Anima/qwen_image_vae.safetensors":{"hash":"","size":253806246},"2_Anima\\animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360},"2_Anima/animaOfficial_preview3Base.safetensors":{"hash":"","size":4182218360}},"ds":{"scale":0.8142857142857145,"offset":[-324.84628826554604,420.82336716169186]}},"version":0.4} \ No newline at end of file diff --git a/jzl_workflows/data/anima_t2i_upscale.json b/jzl_workflows/data/anima_t2i_upscale.json new file mode 100644 index 0000000..87b4bff --- /dev/null +++ b/jzl_workflows/data/anima_t2i_upscale.json @@ -0,0 +1,1895 @@ +{ + "id": "aaeefe57-e932-4ecf-9c14-720bce8c11f5", + "revision": 0, + "last_node_id": 36, + "last_link_id": 50, + "nodes": [ + { + "id": 11, + "type": "Reroute", + "pos": [ + 1660, + 880 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 12, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "widget": { + "name": "value" + }, + "link": 6 + } + ], + "outputs": [ + { + "name": "", + "type": "INT", + "links": [ + 14 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 12, + "type": "Reroute", + "pos": [ + 1660, + 910 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 13, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "widget": { + "name": "value" + }, + "link": 7 + } + ], + "outputs": [ + { + "name": "", + "type": "INT", + "links": [ + 15 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 13, + "type": "Reroute", + "pos": [ + 1660, + 850 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 11, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "widget": { + "name": "value" + }, + "link": 8 + } + ], + "outputs": [ + { + "name": "", + "type": "INT", + "links": [ + 13 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 18, + "type": "Reroute", + "pos": [ + 1660, + 1030 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 9, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 10 + } + ], + "outputs": [ + { + "name": "", + "type": "VAE", + "links": [ + 11 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 21, + "type": "EmptySD3LatentImage", + "pos": [ + 1770, + 910 + ], + "size": [ + 210, + 106 + ], + "flags": { + "collapsed": true, + "pinned": true + }, + "order": 18, + "mode": 0, + "inputs": [ + { + "localized_name": "宽度", + "name": "width", + "type": "INT", + "widget": { + "name": "width" + }, + "link": 13 + }, + { + "localized_name": "高度", + "name": "height", + "type": "INT", + "widget": { + "name": "height" + }, + "link": 14 + }, + { + "localized_name": "批量大小", + "name": "batch_size", + "type": "INT", + "widget": { + "name": "batch_size" + }, + "link": 15 + } + ], + "outputs": [ + { + "localized_name": "Latent", + "name": "LATENT", + "type": "LATENT", + "slot_index": 0, + "links": [ + 16 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "EmptySD3LatentImage", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + 640, + 1280, + 1 + ], + "color": "#322", + "bgcolor": "#533" + }, + { + "id": 14, + "type": "CLIPLoader", + "pos": [ + 1200, + 470 + ], + "size": [ + 430, + 160 + ], + "flags": { + "pinned": true + }, + "order": 0, + "mode": 0, + "inputs": [ + { + "localized_name": "CLIP名称", + "name": "clip_name", + "type": "COMBO", + "widget": { + "name": "clip_name" + }, + "link": null + }, + { + "localized_name": "类型", + "name": "type", + "type": "COMBO", + "widget": { + "name": "type" + }, + "link": null + }, + { + "localized_name": "设备", + "name": "device", + "shape": 7, + "type": "COMBO", + "widget": { + "name": "device" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "CLIP", + "name": "CLIP", + "type": "CLIP", + "links": [ + 9 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "CLIPLoader", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "Anima\\qwen_3_06b_base.safetensors", + "stable_diffusion", + "default" + ], + "color": "#323", + "bgcolor": "#535" + }, + { + "id": 17, + "type": "VAELoader", + "pos": [ + 1200, + 1030 + ], + "size": [ + 430, + 90 + ], + "flags": { + "pinned": true + }, + "order": 1, + "mode": 0, + "inputs": [ + { + "localized_name": "vae名称", + "name": "vae_name", + "type": "COMBO", + "widget": { + "name": "vae_name" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "VAE", + "name": "VAE", + "type": "VAE", + "links": [ + 10 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "VAELoader", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "Anima\\qwen_image_vae.safetensors" + ], + "color": "#323", + "bgcolor": "#535" + }, + { + "id": 25, + "type": "Note", + "pos": [ + 2490, + 570 + ], + "size": [ + 290, + 460 + ], + "flags": { + "pinned": true + }, + "order": 2, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": { + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "简单的测试工作流\n\nVAE解码分块根据显存大小调节，不溢出为原则\n\n一次需要生成多张图片，调整空图像的批量大小\n\n\n\n" + ], + "color": "#432", + "bgcolor": "#653" + }, + { + "id": 15, + "type": "Reroute", + "pos": [ + 1660, + 470 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 8, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 9 + } + ], + "outputs": [ + { + "name": "", + "type": "CLIP", + "links": [ + 5, + 24 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000", + "shape": "box" + }, + { + "id": 7, + "type": "XB_CanvasLabel", + "pos": [ + 2850, + -20 + ], + "size": [ + 460.5126953125, + 37 + ], + "flags": { + "pinned": true, + "allow_interaction": false + }, + "order": 3, + "mode": 0, + "inputs": [], + "outputs": [], + "title": "更多教程全网搜：机智罗_LX", + "properties": { + "fontSize": 37, + "fontColor": "#ffffff", + "textAlign": "left", + "backgroundColor": "transparent", + "padding": 0, + "borderRadius": 0, + "angle": 0, + "aux_id": "wjluoxiao/XB_ToolBox", + "Node name for S&R": "XB_CanvasLabel" + }, + "color": "#fff0", + "bgcolor": "#fff0" + }, + { + "id": 26, + "type": "CLIPTextEncode", + "pos": [ + 1750, + 790 + ], + "size": [ + 281.48203125, + 88 + ], + "flags": { + "pinned": true + }, + "order": 16, + "mode": 0, + "inputs": [ + { + "localized_name": "clip", + "name": "clip", + "type": "CLIP", + "link": 24 + }, + { + "localized_name": "文本", + "name": "text", + "type": "STRING", + "widget": { + "name": "text" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "条件", + "name": "CONDITIONING", + "type": "CONDITIONING", + "links": [ + 33 + ] + } + ], + "title": "CLIP Text Encode (Negative Prompt)", + "properties": { + "cnr_id": "comfy-core", + "ver": "0.3.65", + "Node name for S&R": "CLIPTextEncode", + "enableTabs": false, + "tabWidth": 65, + "tabXOffset": 10, + "hasSecondTab": false, + "secondTabText": "Send Back", + "secondTabOffset": 80, + "secondTabWidth": 65 + }, + "widgets_values": [ + "worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia" + ], + "color": "#322", + "bgcolor": "#533" + }, + { + "id": 22, + "type": "Reroute", + "pos": [ + 2070, + 880 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 20, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 16 + } + ], + "outputs": [ + { + "name": "", + "type": "LATENT", + "links": [ + 34 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 19, + "type": "Reroute", + "pos": [ + 2320, + 1030 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 17, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 11 + } + ], + "outputs": [ + { + "name": "", + "type": "VAE", + "links": [ + 35 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000" + }, + { + "id": 2, + "type": "UNETLoader", + "pos": [ + 1200, + 290 + ], + "size": [ + 430, + 120 + ], + "flags": { + "pinned": true + }, + "order": 4, + "mode": 0, + "inputs": [ + { + "localized_name": "UNet名称", + "name": "unet_name", + "type": "COMBO", + "widget": { + "name": "unet_name" + }, + "link": null + }, + { + "localized_name": "数据类型", + "name": "weight_dtype", + "type": "COMBO", + "widget": { + "name": "weight_dtype" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "模型", + "name": "MODEL", + "type": "MODEL", + "links": [ + 26 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "UNETLoader", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "Anima\\animaOfficial_preview3Base.safetensors", + "default" + ], + "color": "#323", + "bgcolor": "#535" + }, + { + "id": 35, + "type": "Reroute", + "pos": [ + 1660, + 130 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 14, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 46 + } + ], + "outputs": [ + { + "name": "", + "type": "UPSCALE_MODEL", + "links": [ + 45 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000", + "shape": "box" + }, + { + "id": 29, + "type": "XB_ROCmKSampler", + "pos": [ + 2190, + 290 + ], + "size": [ + 240, + 560 + ], + "flags": { + "pinned": true + }, + "order": 21, + "mode": 0, + "inputs": [ + { + "localized_name": "模型", + "name": "model", + "type": "MODEL", + "link": 31 + }, + { + "localized_name": "正面提示词", + "name": "positive", + "type": "CONDITIONING", + "link": 32 + }, + { + "localized_name": "负面提示词", + "name": "negative", + "type": "CONDITIONING", + "link": 33 + }, + { + "localized_name": "潜变量", + "name": "latent", + "type": "LATENT", + "link": 34 + }, + { + "localized_name": "随机种子", + "name": "seed", + "type": "INT", + "widget": { + "name": "seed" + }, + "link": null + }, + { + "localized_name": "采样步数", + "name": "steps", + "type": "INT", + "widget": { + "name": "steps" + }, + "link": null + }, + { + "localized_name": "引导系数(CFG)", + "name": "cfg", + "type": "FLOAT", + "widget": { + "name": "cfg" + }, + "link": null + }, + { + "localized_name": "采样器", + "name": "sampler", + "type": "COMBO", + "widget": { + "name": "sampler" + }, + "link": null + }, + { + "localized_name": "调度器", + "name": "scheduler", + "type": "COMBO", + "widget": { + "name": "scheduler" + }, + "link": null + }, + { + "localized_name": "降噪强度", + "name": "denoise", + "type": "FLOAT", + "widget": { + "name": "denoise" + }, + "link": null + }, + { + "localized_name": "采样后清理", + "name": "cleanup", + "type": "COMBO", + "widget": { + "name": "cleanup" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "Latent", + "name": "LATENT", + "type": "LATENT", + "links": [ + 28 + ] + } + ], + "properties": { + "Node name for S&R": "XB_ROCmKSampler" + }, + "widgets_values": [ + 542421360775416, + "fixed", + 25, + 4, + "res_multistep", + "simple", + 1, + "不做任何清理" + ] + }, + { + "id": 16, + "type": "XB_ImageParamsMaster", + "pos": [ + 1200, + 690 + ], + "size": [ + 430, + 278 + ], + "flags": { + "pinned": true + }, + "order": 5, + "mode": 0, + "inputs": [ + { + "localized_name": "画幅比例", + "name": "aspect_ratio", + "type": "COMBO", + "widget": { + "name": "aspect_ratio" + }, + "link": null + }, + { + "localized_name": "宽度", + "name": "width", + "type": "INT", + "widget": { + "name": "width" + }, + "link": null + }, + { + "localized_name": "高度", + "name": "height", + "type": "INT", + "widget": { + "name": "height" + }, + "link": null + }, + { + "localized_name": "生成数量", + "name": "batch_size", + "type": "INT", + "widget": { + "name": "batch_size" + }, + "link": null + }, + { + "localized_name": "强度(小数)", + "name": "strength_float", + "type": "FLOAT", + "widget": { + "name": "strength_float" + }, + "link": null + }, + { + "localized_name": "强度(整数)", + "name": "strength_int", + "type": "INT", + "widget": { + "name": "strength_int" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "图片宽度", + "name": "Image Width", + "type": "INT", + "links": [ + 8 + ] + }, + { + "localized_name": "图片高度", + "name": "Image Height", + "type": "INT", + "links": [ + 6 + ] + }, + { + "localized_name": "生成数量", + "name": "Batch Size", + "type": "INT", + "links": [ + 7 + ] + }, + { + "localized_name": "浮点控制", + "name": "Float Control", + "type": "FLOAT", + "links": null + }, + { + "localized_name": "整数控制", + "name": "Int Control", + "type": "INT", + "links": null + }, + { + "localized_name": "缩放尺寸", + "name": "Scale Size", + "type": "INT", + "links": null + } + ], + "properties": { + "aux_id": "wjluoxiao/XB_ToolBox", + "Node name for S&R": "XB_ImageParamsMaster", + "ver": "67f93345dd8934fee591e04dcea441b258614a4c" + }, + "widgets_values": [ + "9:16", + 720, + 1280, + 1, + 1, + 1 + ], + "color": "#323", + "bgcolor": "#535" + }, + { + "id": 30, + "type": "XB_ROCmVAEDecode", + "pos": [ + 2490, + 290 + ], + "size": [ + 236.3169921875, + 126 + ], + "flags": { + "pinned": true + }, + "order": 22, + "mode": 0, + "inputs": [ + { + "localized_name": "潜变量", + "name": "samples", + "type": "LATENT", + "link": 28 + }, + { + "localized_name": "VAE模型", + "name": "vae", + "type": "VAE", + "link": 35 + }, + { + "localized_name": "空间分块(0=自动)", + "name": "tile", + "type": "INT", + "widget": { + "name": "tile" + }, + "link": null + }, + { + "localized_name": "分块重叠(0=自动)", + "name": "overlap", + "type": "INT", + "widget": { + "name": "overlap" + }, + "link": null + }, + { + "localized_name": "清理力度", + "name": "cleanup", + "type": "COMBO", + "widget": { + "name": "cleanup" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "图像", + "name": "IMAGE", + "type": "IMAGE", + "links": [ + 44, + 50 + ] + } + ], + "properties": { + "Node name for S&R": "XB_ROCmVAEDecode" + }, + "widgets_values": [ + 320, + 32, + "单次缓存清理" + ] + }, + { + "id": 34, + "type": "Reroute", + "pos": [ + 2640, + 130 + ], + "size": [ + 75, + 26 + ], + "flags": { + "pinned": true + }, + "order": 19, + "mode": 0, + "inputs": [ + { + "name": "", + "type": "*", + "link": 45 + } + ], + "outputs": [ + { + "name": "", + "type": "UPSCALE_MODEL", + "links": [ + 39, + 40 + ] + } + ], + "properties": { + "showOutputText": false, + "horizontal": false, + "ue_properties": { + "version": "7.7", + "widget_ue_connectable": {}, + "input_ue_unconnectable": {} + } + }, + "color": "#222", + "bgcolor": "#000", + "shape": "box" + }, + { + "id": 31, + "type": "ImageUpscaleWithModel", + "pos": [ + 2740, + 210 + ], + "size": [ + 151.9, + 46 + ], + "flags": { + "collapsed": false, + "pinned": true + }, + "order": 23, + "mode": 0, + "inputs": [ + { + "localized_name": "放大模型", + "name": "upscale_model", + "type": "UPSCALE_MODEL", + "link": 39 + }, + { + "localized_name": "图像", + "name": "image", + "type": "IMAGE", + "link": 44 + } + ], + "outputs": [ + { + "localized_name": "图像", + "name": "IMAGE", + "type": "IMAGE", + "links": [ + 38 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.3.59", + "Node name for S&R": "ImageUpscaleWithModel" + }, + "widgets_values": [], + "color": "#232", + "bgcolor": "#353" + }, + { + "id": 33, + "type": "ImageUpscaleWithModel", + "pos": [ + 2900, + 130 + ], + "size": [ + 151.9, + 46 + ], + "flags": { + "collapsed": false, + "pinned": true + }, + "order": 24, + "mode": 0, + "inputs": [ + { + "localized_name": "放大模型", + "name": "upscale_model", + "type": "UPSCALE_MODEL", + "link": 40 + }, + { + "localized_name": "图像", + "name": "image", + "type": "IMAGE", + "link": 38 + } + ], + "outputs": [ + { + "localized_name": "图像", + "name": "IMAGE", + "type": "IMAGE", + "links": [ + 42, + 43, + 49 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.3.59", + "Node name for S&R": "ImageUpscaleWithModel" + }, + "widgets_values": [], + "color": "#232", + "bgcolor": "#353" + }, + { + "id": 20, + "type": "SaveImage", + "pos": [ + 3080, + 130 + ], + "size": [ + 230, + 320 + ], + "flags": { + "pinned": true + }, + "order": 25, + "mode": 0, + "inputs": [ + { + "localized_name": "图片", + "name": "images", + "type": "IMAGE", + "link": 42 + }, + { + "localized_name": "文件名前缀", + "name": "filename_prefix", + "type": "STRING", + "widget": { + "name": "filename_prefix" + }, + "link": null + } + ], + "outputs": [], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "SaveImage", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "图片/Anima/文生图/%date:yyyy-MM-dd%/Anima-文生图" + ], + "color": "#223", + "bgcolor": "#335" + }, + { + "id": 9, + "type": "XB_CanvasLabel", + "pos": [ + 1170, + -210 + ], + "size": [ + 2164.013671875, + 158 + ], + "flags": { + "pinned": true, + "allow_interaction": false + }, + "order": 6, + "mode": 0, + "inputs": [], + "outputs": [], + "title": "Anima 文生图基础+放大工作流", + "properties": { + "fontSize": 158, + "fontColor": "#ffffff", + "textAlign": "left", + "backgroundColor": "transparent", + "padding": 0, + "borderRadius": 0, + "angle": 0, + "aux_id": "wjluoxiao/XB_ToolBox", + "Node name for S&R": "XB_CanvasLabel" + }, + "color": "#fff0", + "bgcolor": "#fff0" + }, + { + "id": 32, + "type": "UpscaleModelLoader", + "pos": [ + 1200, + 130 + ], + "size": [ + 430, + 90 + ], + "flags": { + "collapsed": false, + "pinned": true + }, + "order": 7, + "mode": 0, + "inputs": [ + { + "localized_name": "模型名称", + "name": "model_name", + "type": "COMBO", + "widget": { + "name": "model_name" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "放大模型", + "name": "UPSCALE_MODEL", + "type": "UPSCALE_MODEL", + "links": [ + 46 + ] + } + ], + "properties": { + "cnr_id": "comfy-core", + "ver": "0.3.59", + "Node name for S&R": "UpscaleModelLoader" + }, + "widgets_values": [ + "2x_AnimeJaNai_HD_V3_Compact.pth" + ], + "color": "#323", + "bgcolor": "#535" + }, + { + "id": 28, + "type": "XB_SageAttentionAccelerator", + "pos": [ + 1750, + 290 + ], + "size": [ + 280, + 60 + ], + "flags": { + "pinned": true + }, + "order": 10, + "mode": 4, + "inputs": [ + { + "localized_name": "模型", + "name": "model", + "type": "MODEL", + "link": 26 + }, + { + "localized_name": "加速档位预设", + "name": "preset", + "type": "COMBO", + "widget": { + "name": "preset" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "模型", + "name": "MODEL", + "type": "MODEL", + "links": [ + 31 + ] + } + ], + "properties": { + "aux_id": "WJLUOXIAO/XB_ToolBox", + "Node name for S&R": "XB_SageAttentionAccelerator", + "ver": "2b291a666b781c0b10117bc51e5edf286ffda5a5" + }, + "widgets_values": [ + "内置模式 C (128x16x16)" + ] + }, + { + "id": 1, + "type": "XB_PlaySound", + "pos": [ + 2930, + 360 + ], + "size": [ + 300, + 40 + ], + "flags": { + "collapsed": true, + "pinned": true + }, + "order": 26, + "mode": 0, + "inputs": [ + { + "label": "输入", + "localized_name": "输入", + "name": "any", + "type": "*", + "link": 43 + }, + { + "label": "模式", + "localized_name": "模式", + "name": "mode", + "type": "COMBO", + "widget": { + "name": "mode" + }, + "link": null + }, + { + "label": "音量", + "localized_name": "音量", + "name": "volume", + "type": "FLOAT", + "widget": { + "name": "volume" + }, + "link": null + }, + { + "localized_name": "file", + "name": "file", + "type": "STRING", + "widget": { + "name": "file" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "*", + "name": "*", + "shape": 6, + "type": "*" + } + ], + "title": "提示音📢", + "properties": { + "cnr_id": "XB_ToolBox", + "ver": "f2838ed5e59de4d73cde5c98354b87a8d3200190", + "Node name for S&R": "XB_PlaySound", + "aux_id": "wjluoxiao/XB_ToolBox", + "ue_properties": { + "widget_ue_connectable": {}, + "input_ue_unconnectable": {}, + "version": "7.3" + }, + "widget_ue_connectable": {} + }, + "widgets_values": [ + "always", + 1, + "notify.mp3" + ], + "color": "#223", + "bgcolor": "#335" + }, + { + "id": 36, + "type": "Image Comparer (rgthree)", + "pos": [ + 2830, + 510 + ], + "size": [ + 470, + 600 + ], + "flags": {}, + "order": 27, + "mode": 0, + "inputs": [ + { + "dir": 3, + "label": "图像_A", + "localized_name": "图像_A", + "name": "image_a", + "type": "IMAGE", + "link": 49 + }, + { + "dir": 3, + "label": "图像_B", + "localized_name": "图像_B", + "name": "image_b", + "type": "IMAGE", + "link": 50 + } + ], + "outputs": [], + "properties": { + "cnr_id": "rgthree-comfy", + "ver": "1.0.2604070017", + "comparer_mode": "Slide" + }, + "widgets_values": [ + [ + { + "name": "A", + "selected": true, + "url": "/api/view?filename=rgthree.compare._temp_crjut_00007_.png&type=temp&subfolder=&rand=0.17847560995259604" + }, + { + "name": "B", + "selected": true, + "url": "/api/view?filename=rgthree.compare._temp_crjut_00008_.png&type=temp&subfolder=&rand=0.5451640759696612" + } + ] + ] + }, + { + "id": 6, + "type": "CLIPTextEncode", + "pos": [ + 1750, + 410 + ], + "size": [ + 280, + 320 + ], + "flags": { + "pinned": true + }, + "order": 15, + "mode": 0, + "inputs": [ + { + "localized_name": "clip", + "name": "clip", + "type": "CLIP", + "link": 5 + }, + { + "localized_name": "文本", + "name": "text", + "type": "STRING", + "widget": { + "name": "text" + }, + "link": null + } + ], + "outputs": [ + { + "localized_name": "条件", + "name": "CONDITIONING", + "type": "CONDITIONING", + "slot_index": 0, + "links": [ + 32 + ] + } + ], + "title": "CLIP Text Encode (Positive Prompt)", + "properties": { + "cnr_id": "comfy-core", + "ver": "0.16.3", + "Node name for S&R": "CLIPTextEncode", + "ue_properties": { + "widget_ue_connectable": {}, + "version": "7.7", + "input_ue_unconnectable": {} + } + }, + "widgets_values": [ + "(masterpiece, best quality, very aesthetic, ultra detailed)\n\nintricate details, 4k,\n\nlong hair, double bun, twintails, parted bangs, tiara, earrings, blue eyes, red choker, blue sailor collar, red bow, white shirt, elbow gloves, white gloves, blue skirt, standing, cowboy shot, city, outdoors, night, moon, milky way, reaching out, smile" + ], + "color": "#232", + "bgcolor": "#353" + } + ], + "links": [ + [ + 5, + 15, + 0, + 6, + 0, + "CLIP" + ], + [ + 6, + 16, + 1, + 11, + 0, + "INT" + ], + [ + 7, + 16, + 2, + 12, + 0, + "INT" + ], + [ + 8, + 16, + 0, + 13, + 0, + "INT" + ], + [ + 9, + 14, + 0, + 15, + 0, + "CLIP" + ], + [ + 10, + 17, + 0, + 18, + 0, + "VAE" + ], + [ + 11, + 18, + 0, + 19, + 0, + "VAE" + ], + [ + 13, + 13, + 0, + 21, + 0, + "INT" + ], + [ + 14, + 11, + 0, + 21, + 1, + "INT" + ], + [ + 15, + 12, + 0, + 21, + 2, + "INT" + ], + [ + 16, + 21, + 0, + 22, + 0, + "LATENT" + ], + [ + 24, + 15, + 0, + 26, + 0, + "CLIP" + ], + [ + 26, + 2, + 0, + 28, + 0, + "MODEL" + ], + [ + 28, + 29, + 0, + 30, + 0, + "LATENT" + ], + [ + 31, + 28, + 0, + 29, + 0, + "MODEL" + ], + [ + 32, + 6, + 0, + 29, + 1, + "CONDITIONING" + ], + [ + 33, + 26, + 0, + 29, + 2, + "CONDITIONING" + ], + [ + 34, + 22, + 0, + 29, + 3, + "LATENT" + ], + [ + 35, + 19, + 0, + 30, + 1, + "VAE" + ], + [ + 38, + 31, + 0, + 33, + 1, + "IMAGE" + ], + [ + 39, + 34, + 0, + 31, + 0, + "UPSCALE_MODEL" + ], + [ + 40, + 34, + 0, + 33, + 0, + "UPSCALE_MODEL" + ], + [ + 42, + 33, + 0, + 20, + 0, + "IMAGE" + ], + [ + 43, + 33, + 0, + 1, + 0, + "IMAGE" + ], + [ + 44, + 30, + 0, + 31, + 1, + "IMAGE" + ], + [ + 45, + 35, + 0, + 34, + 0, + "*" + ], + [ + 46, + 32, + 0, + 35, + 0, + "UPSCALE_MODEL" + ], + [ + 49, + 33, + 0, + 36, + 0, + "IMAGE" + ], + [ + 50, + 30, + 0, + 36, + 1, + "IMAGE" + ] + ], + "groups": [ + { + "id": 1, + "title": "Anima 文生图基础+放大-机智罗", + "bounding": [ + 1170, + 40, + 2170, + 1110 + ], + "color": "#3f789e", + "font_size": 24, + "flags": { + "pinned": true + } + } + ], + "config": {}, + "extra": { + "ds": { + "scale": 0.693433494944134, + "offset": [ + -429.5808249298941, + 503.6147890270224 + ] + } + }, + "version": 0.4 +} \ No newline at end of file diff --git a/jzl_workflows/data/index.json b/jzl_workflows/data/index.json new file mode 100644 index 0000000..1a15a8b --- /dev/null +++ b/jzl_workflows/data/index.json @@ -0,0 +1,55 @@ +[ + { + "moduleName": "jzl", + "category": "JZL Workflows", + "title": "Anima", + "icon": "icon-[lucide--image]", + "type": "image", + "templates": [ + { + "name": "anima_t2i_basic", + "title": "Anima Text to Image (Basic)", + "description": "Anima basic text-to-image: prompt preset + model loader + sampling/decoding in one graph.", + "tags": ["Anima", "Text to Image"], + "thumbnail": ["anima_t2i_basic-1.webp"], + "date": "2026-11-09", + "username": "JZL", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_inpaint_basic", + "title": "Anima Inpainting (Basic)", + "description": "Anima inpainting: basic masked-region repaint flow.", + "tags": ["Anima", "Inpainting"], + "thumbnail": ["anima_inpaint_basic-1.webp"], + "date": "2026-11-09", + "username": "JZL", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_t2i_upscale", + "title": "Anima Text to Image + Upscale", + "description": "Anima text-to-image followed by an upscale pass.", + "tags": ["Anima", "Text to Image", "Upscale"], + "thumbnail": ["anima_t2i_upscale-1.webp"], + "date": "2026-11-09", + "username": "JZL", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_control_t2i", + "title": "Anima Control to Image (Basic)", + "description": "Anima control-guided generation: basic flow with control conditions.", + "tags": ["Anima", "Control"], + "thumbnail": ["anima_control_t2i-1.webp"], + "date": "2026-11-09", + "username": "JZL", + "openSource": true, + "minComfyUIVersion": "0.0.0" + } + ] + } +] diff --git a/jzl_workflows/data/index.zh.json b/jzl_workflows/data/index.zh.json new file mode 100644 index 0000000..6cda8d6 --- /dev/null +++ b/jzl_workflows/data/index.zh.json @@ -0,0 +1,55 @@ +[ + { + "moduleName": "jzl", + "category": "机智罗的工作流", + "title": "Anima", + "icon": "icon-[lucide--image]", + "type": "image", + "templates": [ + { + "name": "anima_t2i_basic", + "title": "Anima 文生图基础工作流", + "description": "Anima 基础文生图：提示词预设 + 模型加载 + 采样解码一体化。", + "tags": ["Anima", "文生图"], + "thumbnail": ["anima_t2i_basic-1.webp"], + "date": "2026-11-09", + "username": "机智罗", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_inpaint_basic", + "title": "Anima 局部重绘基础工作流", + "description": "Anima 局部重绘：遮罩区域重绘的基础流程。", + "tags": ["Anima", "局部重绘"], + "thumbnail": ["anima_inpaint_basic-1.webp"], + "date": "2026-11-09", + "username": "机智罗", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_t2i_upscale", + "title": "Anima 文生图基础+放大工作流", + "description": "Anima 文生图后接放大流程。", + "tags": ["Anima", "文生图", "放大"], + "thumbnail": ["anima_t2i_upscale-1.webp"], + "date": "2026-11-09", + "username": "机智罗", + "openSource": true, + "minComfyUIVersion": "0.0.0" + }, + { + "name": "anima_control_t2i", + "title": "Anima 控制生图基础工作流", + "description": "Anima 控制生图：带控制条件的基础流程。", + "tags": ["Anima", "控制生图"], + "thumbnail": ["anima_control_t2i-1.webp"], + "date": "2026-11-09", + "username": "机智罗", + "openSource": true, + "minComfyUIVersion": "0.0.0" + } + ] + } +] diff --git a/jzl_workflows/routes.py b/jzl_workflows/routes.py new file mode 100644 index 0000000..864819b --- /dev/null +++ b/jzl_workflows/routes.py @@ -0,0 +1,114 @@ +"""XB_ToolBox — 「机智罗的工作流」模板数据服务 +========================================================= +清单结构照抄官方 ``comfyui-workflow-templates`` 的 ``index.json`` / ``index.zh.json``： +``[{moduleName, category, title, icon, type, templates: [...]}]``。 + +前端「机智罗」侧栏面板只读这两个清单，按 ``category`` / ``title`` 分组渲染卡片， +点击卡片再取 ``.json`` 加载工作流，预览图按官方 ``-1.webp`` 约定托管。 + +安全：只放行清单里登记过的文件名，且解析后的路径必须仍在 ``xb_templates`` 目录内。 +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path + +from aiohttp import web + +ROOT = Path(__file__).resolve().parent / "data" + +# 面板静态资源（图标 / 样式）：由本模块单独托管，绕开 /extensions 的 24h 强缓存 +WEB_DIR = Path(__file__).resolve().parent / "web" +WEB_ASSETS = ( + "xb_icon_dark.png", + "xb_icon_light.png", + "xb_workflow_gallery.css", +) + +# 清单文件名（英文 / 中文） +MANIFESTS = ("index.json", "index.zh.json") + +# 放行文件名只允许 ASCII 安全字符，天然排除 "/" 与 ".." +_SAFE_NAME = re.compile(r"^[A-Za-z0-9._-]+$") + + +def _read_manifest(name: str) -> list: + path = ROOT / name + if not path.is_file(): + return [] + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return [] + return data if isinstance(data, list) else [] + + +def _allowed_files() -> set[str]: + """清单登记过的文件名白名单（含两个 index 清单本身）。""" + allowed = set(MANIFESTS) + for manifest_name in MANIFESTS: + for group in _read_manifest(manifest_name): + if not isinstance(group, dict): + continue + templates = group.get("templates") + if not isinstance(templates, list): + continue + for tpl in templates: + if not isinstance(tpl, dict): + continue + name = tpl.get("name") + if isinstance(name, str) and _SAFE_NAME.match(name): + allowed.add(f"{name}.json") + thumbnails = tpl.get("thumbnail") + if isinstance(thumbnails, list): + for thumb in thumbnails: + if not isinstance(thumb, str): + continue + base = thumb.replace("\\", "/").rsplit("/", 1)[-1] + if _SAFE_NAME.match(base): + allowed.add(base) + return allowed + + +def register_routes(): + from server import PromptServer + + routes = PromptServer.instance.routes + + @routes.get("/xb_toolbox/templates/{filename}") + async def get_template_asset(request): + filename = request.match_info.get("filename", "") + if not _SAFE_NAME.match(filename) or filename not in _allowed_files(): + raise web.HTTPNotFound() + + target = (ROOT / filename).resolve() + if target.parent != ROOT or not target.is_file(): + raise web.HTTPNotFound() + + is_manifest = filename in MANIFESTS + headers = { + # 清单必须实时反映磁盘改动，图片只做协商缓存 + "Cache-Control": "no-store" if is_manifest else "no-cache", + "X-Content-Type-Options": "nosniff", + } + return web.FileResponse(target, headers=headers) + + @routes.get("/xb_toolbox/assets/{name}") + async def get_web_asset(request): + """面板静态资源（图标 / 样式）单独托管。 + + /extensions/* 由 ComfyUI 静态托管并带 ``max-age=86400``，换图换样式后浏览器会一直用旧的； + 这里用 no-cache，让浏览器每次协商，改完普通刷新即可生效。 + """ + name = request.match_info.get("name", "") + if name not in WEB_ASSETS: + raise web.HTTPNotFound() + target = WEB_DIR / name + if not target.is_file(): + raise web.HTTPNotFound() + return web.FileResponse( + target, + headers={"Cache-Control": "no-cache", "X-Content-Type-Options": "nosniff"}, + ) diff --git a/jzl_workflows/web/xb_icon_dark.png b/jzl_workflows/web/xb_icon_dark.png new file mode 100644 index 0000000..400d51c Binary files /dev/null and b/jzl_workflows/web/xb_icon_dark.png differ diff --git a/jzl_workflows/web/xb_icon_light.png b/jzl_workflows/web/xb_icon_light.png new file mode 100644 index 0000000..5eb3141 Binary files /dev/null and b/jzl_workflows/web/xb_icon_light.png differ diff --git a/jzl_workflows/web/xb_workflow_gallery.css b/jzl_workflows/web/xb_workflow_gallery.css new file mode 100644 index 0000000..1221f4b --- /dev/null +++ b/jzl_workflows/web/xb_workflow_gallery.css @@ -0,0 +1,404 @@ +/* XB_ToolBox — 「机智罗的工作流」侧栏面板样式 + * 全部走 ComfyUI 的 CSS 变量，跟随明/暗主题。 */ + +.xb-wf-root { + display: flex; + flex-direction: column; + gap: 8px; + height: 100%; + min-height: 0; + box-sizing: border-box; + padding: 10px; + overflow-y: auto; + font-size: 13px; + line-height: 1.4; + color: var(--input-text, #ddd); + background: var(--comfy-menu-bg, #1e1e1e); +} + +.xb-wf-root.is-busy { + cursor: progress; +} + +.xb-wf-head { + display: flex; + align-items: center; + gap: 8px; + flex: 0 0 auto; +} + +.xb-wf-logo { + width: 22px; + height: 22px; + object-fit: contain; + flex: 0 0 auto; +} + +.xb-wf-title { + flex: 1 1 auto; + min-width: 0; + font-weight: 600; + font-size: 13px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.xb-wf-icon-btn { + flex: 0 0 auto; + width: 24px; + height: 24px; + display: inline-flex; + align-items: center; + justify-content: center; + padding: 0; + border: 1px solid var(--border-color, #444); + border-radius: 6px; + background: transparent; + color: inherit; + font-size: 13px; + cursor: pointer; + transition: background-color 0.15s ease; +} + +.xb-wf-icon-btn:hover:not(:disabled) { + background: var(--comfy-input-bg, rgba(255, 255, 255, 0.08)); +} + +.xb-wf-icon-btn:disabled { + opacity: 0.5; + cursor: default; +} + +.xb-wf-search { + flex: 0 0 auto; +} + +.xb-wf-search input { + width: 100%; + box-sizing: border-box; + padding: 5px 8px; + border: 1px solid var(--border-color, #444); + border-radius: 6px; + background: var(--comfy-input-bg, #2a2a2a); + color: var(--input-text, #ddd); + font-size: 12px; + outline: none; +} + +.xb-wf-search input:focus { + border-color: var(--p-primary-color, #4a9eff); +} + +.xb-wf-cats { + display: flex; + flex-wrap: wrap; + gap: 6px; + flex: 0 0 auto; +} + +.xb-wf-cats.is-hidden { + display: none; +} + +.xb-wf-chip { + padding: 3px 9px; + border: 1px solid var(--border-color, #444); + border-radius: 999px; + background: transparent; + color: inherit; + font-size: 12px; + cursor: pointer; + white-space: nowrap; + transition: background-color 0.15s ease, border-color 0.15s ease; +} + +.xb-wf-chip:hover { + background: var(--comfy-input-bg, rgba(255, 255, 255, 0.08)); +} + +.xb-wf-chip.is-active { + border-color: var(--p-primary-color, #4a9eff); + background: var(--p-primary-color, #4a9eff); + color: #fff; +} + +.xb-wf-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(140px, 1fr)); + gap: 8px; + flex: 1 1 auto; + align-content: start; +} + +.xb-wf-card { + display: flex; + flex-direction: column; + gap: 4px; + min-width: 0; + padding: 6px; + border: 1px solid var(--border-color, #444); + border-radius: 8px; + background: var(--comfy-input-bg, #252525); + color: inherit; + font: inherit; + text-align: left; + cursor: pointer; + transition: border-color 0.15s ease, transform 0.15s ease; +} + +.xb-wf-card:hover { + border-color: var(--p-primary-color, #4a9eff); + transform: translateY(-1px); +} + +.xb-wf-card:focus-visible { + outline: 2px solid var(--p-primary-color, #4a9eff); + outline-offset: 1px; +} + +.xb-wf-root.is-busy .xb-wf-card { + pointer-events: none; + opacity: 0.7; +} + +.xb-wf-thumb { + position: relative; + width: 100%; + aspect-ratio: 4 / 3; + overflow: hidden; + border-radius: 5px; + background: var(--comfy-menu-bg, #1a1a1a); +} + +.xb-wf-thumb img { + display: block; + width: 100%; + height: 100%; + object-fit: cover; +} + +.xb-wf-thumb.is-empty { + display: flex; + align-items: center; + justify-content: center; + padding: 6px; + box-sizing: border-box; + background: linear-gradient(135deg, rgba(74, 158, 255, 0.22), rgba(180, 74, 255, 0.22)); +} + +.xb-wf-thumb-text { + font-size: 11px; + line-height: 1.3; + text-align: center; + color: var(--input-text, #ddd); + word-break: break-word; + display: -webkit-box; + -webkit-line-clamp: 3; + -webkit-box-orient: vertical; + overflow: hidden; +} + +.xb-wf-name { + font-size: 12px; + font-weight: 600; + display: -webkit-box; + -webkit-line-clamp: 2; + -webkit-box-orient: vertical; + overflow: hidden; +} + +.xb-wf-tags { + display: flex; + flex-wrap: wrap; + gap: 4px; +} + +.xb-wf-tag { + padding: 1px 6px; + border-radius: 999px; + background: rgba(128, 128, 128, 0.2); + font-size: 10px; + opacity: 0.85; +} + +.xb-wf-empty { + grid-column: 1 / -1; + padding: 24px 12px; + text-align: center; + opacity: 0.6; + font-size: 12px; +} + +.xb-wf-status { + flex: 0 0 auto; + min-height: 0; + font-size: 11px; + opacity: 0.75; +} + +.xb-wf-status:empty { + display: none; +} + +.xb-wf-status.is-error { + opacity: 1; + color: var(--error-text, #ff6b6b); +} + +/* ── 居中大弹窗（与官方「模板」一致的布局）────────────────────────── */ + +.xb-wf-backdrop { + position: fixed; + inset: 0; + z-index: 10000; + display: flex; + align-items: center; + justify-content: center; + padding: 3vh 3vw; + box-sizing: border-box; + background: rgba(0, 0, 0, 0.6); + backdrop-filter: blur(2px); +} + +.xb-wf-backdrop.is-busy { + cursor: progress; +} + +.xb-wf-modal { + display: flex; + width: min(1240px, 100%); + height: min(840px, 100%); + overflow: hidden; + border: 1px solid var(--border-color, #3a3a3a); + border-radius: 12px; + background: var(--comfy-menu-bg, #202020); + color: var(--input-text, #ddd); + font-size: 13px; + line-height: 1.4; + box-shadow: 0 24px 64px rgba(0, 0, 0, 0.55); +} + +.xb-wf-side { + flex: 0 0 208px; + display: flex; + flex-direction: column; + gap: 10px; + padding: 14px 10px; + overflow-y: auto; + border-right: 1px solid var(--border-color, #333); + background: rgba(255, 255, 255, 0.02); +} + +.xb-wf-side-head { + display: flex; + align-items: center; + gap: 8px; + padding: 0 6px 10px; + border-bottom: 1px solid var(--border-color, #333); +} + +.xb-wf-side-title { + font-weight: 600; + font-size: 13px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.xb-wf-side-list { + display: flex; + flex-direction: column; + gap: 2px; +} + +.xb-wf-side-item { + display: block; + width: 100%; + padding: 6px 10px; + border: 0; + border-radius: 6px; + background: transparent; + color: inherit; + font: inherit; + text-align: left; + cursor: pointer; + opacity: 0.85; + transition: background-color 0.12s ease, opacity 0.12s ease; +} + +.xb-wf-side-item:hover { + background: rgba(255, 255, 255, 0.07); + opacity: 1; +} + +.xb-wf-side-item.is-active { + background: var(--p-primary-color, #4a9eff); + color: #fff; + opacity: 1; +} + +.xb-wf-main { + flex: 1 1 auto; + display: flex; + flex-direction: column; + min-width: 0; + min-height: 0; +} + +.xb-wf-main-head { + display: flex; + align-items: center; + gap: 8px; + flex: 0 0 auto; + padding: 12px 14px; + border-bottom: 1px solid var(--border-color, #333); +} + +.xb-wf-search-input { + flex: 1 1 auto; + min-width: 0; + padding: 6px 10px; + border: 1px solid var(--border-color, #444); + border-radius: 8px; + background: var(--comfy-input-bg, #2a2a2a); + color: var(--input-text, #ddd); + font-size: 13px; + outline: none; +} + +.xb-wf-search-input:focus { + border-color: var(--p-primary-color, #4a9eff); +} + +.xb-wf-modal .xb-wf-icon-btn { + width: 28px; + height: 28px; + font-size: 14px; +} + +.xb-wf-modal .xb-wf-grid { + gap: 12px; + padding: 14px; + overflow-y: auto; + grid-template-columns: repeat(auto-fill, minmax(190px, 1fr)); +} + +.xb-wf-modal .xb-wf-card { + padding: 8px; + gap: 6px; +} + +.xb-wf-modal .xb-wf-name { + font-size: 13px; +} + +.xb-wf-modal .xb-wf-status { + padding: 0 14px 10px; +} + +.xb-wf-backdrop.is-busy .xb-wf-card { + pointer-events: none; + opacity: 0.7; +} + diff --git a/cosyvoice3/cosyvoice/cli/__init__.py b/libs/__init__.py similarity index 100% rename from cosyvoice3/cosyvoice/cli/__init__.py rename to libs/__init__.py diff --git a/xb_av.py b/libs/xb_av.py similarity index 100% rename from xb_av.py rename to libs/xb_av.py diff --git a/xb_gallery.py b/libs/xb_gallery.py similarity index 99% rename from xb_gallery.py rename to libs/xb_gallery.py index d13e33c..ae69489 100644 --- a/xb_gallery.py +++ b/libs/xb_gallery.py @@ -37,7 +37,7 @@ MIME_BY_EXT = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ANGLE_ORDER = ("正面", "斜侧", "侧面", "俯视", "仰视") THUMB_MAX = 384 FOLDER_TTL = 5.0 # 普通文件夹扫描缓存（秒） -PACKAGE_ROOT = Path(__file__).resolve().parent +PACKAGE_ROOT = Path(__file__).resolve().parent.parent _cache_lock = threading.Lock() _manifest_cache: dict = {} # {(root, mtime, size): catalogue} diff --git a/xb_paths.py b/libs/xb_paths.py similarity index 100% rename from xb_paths.py rename to libs/xb_paths.py diff --git a/xb_presets.py b/libs/xb_presets.py similarity index 99% rename from xb_presets.py rename to libs/xb_presets.py index f05e03d..0349290 100644 --- a/xb_presets.py +++ b/libs/xb_presets.py @@ -19,7 +19,7 @@ import os import re import time -ROOT_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "xb_skill") +ROOT_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "xb_skill") DEFAULT_NODE = "XB_ImagePromptPreset" ZH = "ZH" diff --git a/xb_skills.py b/libs/xb_skills.py similarity index 99% rename from xb_skills.py rename to libs/xb_skills.py index b1d1e8a..7af6ca5 100644 --- a/xb_skills.py +++ b/libs/xb_skills.py @@ -20,7 +20,7 @@ import re import threading import time -PACKAGE_DIR = os.path.dirname(os.path.abspath(__file__)) +PACKAGE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # 技能文件统一放在 xb_skill/skills（与预设词 txt 同一个 xb_skill 根目录） SKILL_DIR = os.path.join(PACKAGE_DIR, "xb_skill", "skills") # 旧位置（xb_skill/skills）：首次使用时自动把文件迁移过去，老部署无需手动搬 diff --git a/xb_video_server.py b/libs/xb_video_server.py similarity index 100% rename from xb_video_server.py rename to libs/xb_video_server.py diff --git a/yue2_abc.py b/libs/yue2_abc.py similarity index 100% rename from yue2_abc.py rename to libs/yue2_abc.py diff --git a/yue2_styles.py b/libs/yue2_styles.py similarity index 100% rename from yue2_styles.py rename to libs/yue2_styles.py diff --git a/cosyvoice3/cosyvoice/dataset/__init__.py b/nodes/__init__.py similarity index 100% rename from cosyvoice3/cosyvoice/dataset/__init__.py rename to nodes/__init__.py diff --git a/cosyvoice3/cosyvoice/transformer/__init__.py b/nodes/audio/__init__.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/__init__.py rename to nodes/audio/__init__.py diff --git a/nodes_audio_slicer.py b/nodes/audio/nodes_audio_slicer.py similarity index 97% rename from nodes_audio_slicer.py rename to nodes/audio/nodes_audio_slicer.py index 40b324f..57e8137 100644 --- a/nodes_audio_slicer.py +++ b/nodes/audio/nodes_audio_slicer.py @@ -1,485 +1,485 @@ -""" -XB-ToolBox 音频切片节点 -====================== -完全独立的音频加载/切片节点，支持可视化波形截取。 -使用 av (PyAV) 解码，兼容所有格式（同 VHS 用 ffmpeg 的效果） -""" - -import os, asyncio - -import folder_paths -import torch -import av -from aiohttp import web -import server - -from .xb_av import open_av - -def _load_audio_file(filepath: str): - """用 av 解码音频，返回 (waveform_2d, sample_rate)。加载失败返回 None""" - try: - with open_av(filepath) as af: - if not af.streams.audio: - return None - stream = af.streams.audio[0] - sr = stream.codec_context.sample_rate - frames = [] - for frame in af.decode(streams=stream.index): - buf = torch.from_numpy(frame.to_ndarray()) - if buf.dim() == 1: - buf = buf.unsqueeze(0) - elif buf.shape[0] != stream.channels: - buf = buf.view(-1, stream.channels).t() - frames.append(buf) - if not frames: - return None - wav = torch.cat(frames, dim=1) - # 统一转为 float32 - if not wav.dtype.is_floating_point: - if wav.dtype == torch.int16: - wav = wav.float() / (2 ** 15) - elif wav.dtype == torch.int32: - wav = wav.float() / (2 ** 31) - return wav, sr - except Exception: - return None - - -def _make_audio(waveform, sample_rate): - if waveform.dim() == 1: - waveform = waveform.unsqueeze(0).unsqueeze(0) - elif waveform.dim() == 2: - waveform = waveform.unsqueeze(0) - return {"waveform": waveform, "sample_rate": sample_rate} - -@server.PromptServer.instance.routes.post("/xb_audio_waveform") -@server.PromptServer.instance.routes.post("/xb_toolbox/audio_waveform") -async def handle_audio_waveform(request): - """返回音频文件的波形峰值数据，支持按时间范围切片重算""" - try: - data = await request.json() - except Exception: - return web.json_response({"error": "Invalid JSON body"}) - - filename = data.get("filename", "") - num_peaks = data.get("num_peaks", 400) - start_time = data.get("start_time", None) - end_time = data.get("end_time", None) - - if not filename: - return web.json_response({"peaks": [], "duration": 0}) - - input_dir = folder_paths.get_input_directory() - audio_path = os.path.join(input_dir, filename) - - if not os.path.exists(audio_path): - return web.json_response({"peaks": [], "duration": 0}) - - try: - loop = asyncio.get_event_loop() - result = await loop.run_in_executor(None, _load_audio_file, audio_path) - if result is None: - return web.json_response({"peaks": [], "duration": 0}) - waveform, sample_rate = result - if waveform.shape[0] > 1: - waveform = waveform.mean(dim=0, keepdim=True) - duration = waveform.shape[1] / sample_rate - - if start_time is not None and end_time is not None and end_time > start_time: - s_idx = max(0, int(start_time * sample_rate)) - e_idx = min(waveform.shape[1], int(end_time * sample_rate)) - waveform = waveform[:, s_idx:e_idx] - - total_samples = waveform.shape[1] - if total_samples == 0: - return web.json_response({"peaks": [], "duration": duration}) - - chunk_size = max(1, total_samples // num_peaks) - peaks = [] - for i in range(num_peaks): - chunk = waveform[0, i * chunk_size : (i + 1) * chunk_size] - if chunk.numel() == 0: - peaks.append([0.0, 0.0]) - continue - peaks.append([float(chunk.max().item()), float(chunk.min().item())]) - - return web.json_response({"peaks": peaks, "duration": duration}) - except Exception as e: - return web.json_response({"error": str(e)}) - - -def _snap_4n1(frames): - """将帧数对齐到 4N+1（满足 InfiniteTalk 的硬性要求）""" - return max(1, ((frames + 2) // 4) * 4 + 1) - - -# ============================================================ -# XB_AudioSlicer — 音频切片（基础版） -# ============================================================ -class XB_AudioSlicer: - """音频加载与切片节点 — 支持频谱可视化 + 拖拽分割线截取 - 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。 - 额外输出 fps，方便连接到总线统一帧率。""" - - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - audio_files = ["none"] - if os.path.exists(input_dir): - for f in sorted(os.listdir(input_dir)): - if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): - audio_files.append(f) - - return { - "required": { - "audio": (audio_files,), - "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), - "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01, "precision": 2, "round": 0.01}), - "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01, "precision": 2, "round": 0.01}), - "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), - } - } - - RETURN_TYPES = ("AUDIO", "INT", "FLOAT") - RETURN_NAMES = ("audio", "frame_count", "fps") - FUNCTION = "slice_audio" - CATEGORY = "XB_ToolBox/Audio" - - @classmethod - def VALIDATE_INPUTS(cls, audio, **kwargs): - return True - - def slice_audio(self, audio, fps, start_time, end_time, duration_display): - if audio == "none": - return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) - - result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), audio)) - if result is None: - return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) - - waveform, sample_rate = result - if waveform.shape[0] > 1: - waveform = waveform.mean(dim=0, keepdim=True) - - total_duration = waveform.shape[1] / sample_rate - frame_duration = 1.0 / fps - - start_time = max(0.0, min(start_time, total_duration)) - end_time = max(start_time + frame_duration, min(end_time, total_duration)) - - start_sample = int(start_time * sample_rate) - end_sample = int(end_time * sample_rate) - - if end_sample <= start_sample: - end_sample = min(start_sample + int(frame_duration * 4 * sample_rate), waveform.shape[1]) - - sliced = waveform[:, start_sample:end_sample] - duration_sec = (end_sample - start_sample) / sample_rate - raw_frames = int(round(duration_sec * fps)) - frame_count = _snap_4n1(raw_frames) - # 🛡️ 音频物理长度对齐 4N+1 帧数，防止音视频时间轴错位 - target_samples = int((frame_count / fps) * sample_rate) - if sliced.shape[1] < target_samples: - pad = torch.zeros((sliced.shape[0], target_samples - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) - sliced = torch.cat([sliced, pad], dim=1) - elif sliced.shape[1] > target_samples: - sliced = sliced[:, :target_samples] - if frame_count != raw_frames: - print(f"🔧 [音频切片] 帧数 {raw_frames} → {frame_count} (对齐 4N+1)") - return (_make_audio(sliced, sample_rate), frame_count, fps) - - -# ============================================================ -# XB_AudioSlicerV1 — 可视化音频切片（波形+拖拽分割线） -# ============================================================ -class XB_AudioSlicerV1: - """音频切片节点 V1 — 可视化波形窗口 + 拖拽分割线截取 - 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。 - 额外输出 fps，方便连接到总线统一帧率。""" - - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - audio_files = ["none"] - if os.path.exists(input_dir): - for f in sorted(os.listdir(input_dir)): - if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): - audio_files.append(f) - - return { - "required": { - "audio": (audio_files,), - "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), - "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), - } - } - - RETURN_TYPES = ("AUDIO", "INT", "FLOAT") - RETURN_NAMES = ("audio", "frame_count", "fps") - FUNCTION = "slice_audio" - CATEGORY = "XB_ToolBox/Audio" - - @classmethod - def VALIDATE_INPUTS(cls, audio, **kwargs): - return True - - def slice_audio(self, audio, fps, start_time, end_time, duration_display): - if audio == "none": - return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) - - result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), audio)) - if result is None: - return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) - - waveform, sample_rate = result - if waveform.shape[0] > 1: - waveform = waveform.mean(dim=0, keepdim=True) - - total_duration = waveform.shape[1] / sample_rate - frame_duration = 1.0 / fps - start_time = max(0.0, min(start_time, total_duration)) - end_time = max(start_time + frame_duration, min(end_time, total_duration)) - - start_sample = int(start_time * sample_rate) - end_sample = int(end_time * sample_rate) - if end_sample <= start_sample: - end_sample = min(start_sample + int(frame_duration * 4 * sample_rate), waveform.shape[1]) - - sliced = waveform[:, start_sample:end_sample] - duration_sec = (end_sample - start_sample) / sample_rate - raw_frames = int(round(duration_sec * fps)) - frame_count = _snap_4n1(raw_frames) - if frame_count != raw_frames: - print(f"🔧 [音频切片V1] 帧数 {raw_frames} → {frame_count} (对齐 4N+1)") - return (_make_audio(sliced, sample_rate), frame_count, fps) - - -# ============================================================ -# XB_AudioSlicerV2 — 双人音频切片 -# ============================================================ -class XB_AudioSlicerV2: - """双人音频切片 — 音频1 + 间隔(帧) + 音频2，独立可视化 - 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。""" - - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - audio_files = ["none"] - if os.path.exists(input_dir): - for f in sorted(os.listdir(input_dir)): - if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): - audio_files.append(f) - - return { - "required": { - "audio1": (audio_files,), - "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), - "start1": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "end1": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "audio2": (audio_files,), - "start2": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "end2": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "gap_frames": ("INT", {"default": 25, "min": 0, "max": 99999, "step": 1}), - "total_display": ("STRING", {"default": "0 帧", "multiline": False}), - } - } - - RETURN_TYPES = ("AUDIO", "INT", "FLOAT", "AUDIO", "INT", "AUDIO", "INT") - RETURN_NAMES = ("combined_audio", "total_frames", "fps", "audio1", "frames1", "audio2", "frames2") - FUNCTION = "slice_dual" - CATEGORY = "XB_ToolBox/Audio" - - @classmethod - def VALIDATE_INPUTS(cls, audio1, audio2, **kwargs): - return True - - def _load(self, name): - if name == "none": return None, 0 - result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), name)) - if result is None: return None, 0 - wf, sr = result - if wf.shape[0] > 1: wf = wf.mean(dim=0, keepdim=True) - return {"waveform": wf, "sample_rate": sr}, wf.shape[1] / sr - - def _trim(self, audio, td, st, et, sr, fps): - fd = 1.0 / fps - st = max(0.0, min(st, td)); et = max(st + fd, min(et, td)) - ss, es = int(st * sr), int(et * sr) - if es <= ss: es = min(ss + int(fd * 4 * sr), audio.shape[1]) - dur = (es - ss) / sr - raw = int(round(dur * fps)) - fc = _snap_4n1(raw) - # 🛡️ 音频物理长度对齐 4N+1 帧数 - sliced = audio[:, ss:es] - target = int((fc / fps) * sr) - if sliced.shape[1] < target: - pad = torch.zeros((sliced.shape[0], target - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) - sliced = torch.cat([sliced, pad], dim=1) - elif sliced.shape[1] > target: - sliced = sliced[:, :target] - return sliced, fc - - def slice_dual(self, audio1, fps, start1, end1, audio2, start2, end2, gap_frames, total_display): - a1, d1f = self._load(audio1); a2, d2f = self._load(audio2) - sr = 44100 - if a1: sr = a1["sample_rate"] - elif a2: sr = a2["sample_rate"] - - # 各自用原生采样率裁剪 - t1 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f1 = 0 - t2 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f2 = 0 - if a1 and d1f > 0: t1, f1 = self._trim(a1["waveform"], d1f, start1, end1, a1["sample_rate"], fps) - if a2 and d2f > 0: t2, f2 = self._trim(a2["waveform"], d2f, start2, end2, a2["sample_rate"], fps) - - # 统一采样率 - import torchaudio - if a2 and a2["sample_rate"] != sr: - t2 = torchaudio.functional.resample(t2, a2["sample_rate"], sr) - if a1 and a1["sample_rate"] != sr: - t1 = torchaudio.functional.resample(t1, a1["sample_rate"], sr) - - gap_s = int(max(0, gap_frames) / fps * sr) - gap_wf = torch.zeros((1, gap_s), dtype=torch.float32) - combined = torch.cat([t1, gap_wf, t2], dim=1) - total_f = _snap_4n1(f1 + max(0, gap_frames) + f2) - if total_f != f1 + max(0, gap_frames) + f2: - print(f"🔧 [音频切片V2] 合并帧数 {f1 + max(0, gap_frames) + f2} → {total_f} (对齐 4N+1)") - return (_make_audio(combined, sr), total_f, fps, - _make_audio(t1, sr), f1, _make_audio(t2, sr), f2) - - -# ============================================================ -# XB_AudioSlicerV3 — 高级双轨混音与静音切片 (视窗缩放版) -# ============================================================ -class XB_AudioSlicerV3: - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - audio_files = ["none"] - if os.path.exists(input_dir): - for f in sorted(os.listdir(input_dir)): - if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): - audio_files.append(f) - - return { - "required": { - "audio1": (audio_files,), - "fps": ("FLOAT", {"default": 24.0, "min": 1.0, "max": 120.0, "step": 1.0}), - "start1": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 3600.0, "step": 0.01}), - "end1": ("FLOAT", {"default": 10.0, "min": 0.0, "max": 3600.0, "step": 0.01}), - "mute_count1": ("INT", {"default": 0, "min": 0, "max": 20, "step": 1}), - "mutes1_data": ("STRING", {"default": ""}), - "audio2": (audio_files,), - "start2": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 3600.0, "step": 0.01}), - "end2": ("FLOAT", {"default": 10.0, "min": 0.0, "max": 3600.0, "step": 0.01}), - "mute_count2": ("INT", {"default": 0, "min": 0, "max": 20, "step": 1}), - "mutes2_data": ("STRING", {"default": ""}), - "merge_mode": (["接力", "重叠"], {"default": "接力"}), - "total_display": ("STRING", {"default": "0 / 0帧", "multiline": False}), - } - } - - RETURN_TYPES = ("AUDIO", "INT", "FLOAT", "AUDIO", "INT", "AUDIO", "INT") - RETURN_NAMES = ("combined_audio", "total_frames", "fps", "audio1", "frames1", "audio2", "frames2") - FUNCTION = "slice_v3" - CATEGORY = "XB_ToolBox/Audio" - - @classmethod - def VALIDATE_INPUTS(cls, audio1, audio2, **kwargs): - return True - - def _load(self, name): - if name == "none": return None, 0 - result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), name)) - if result is None: return None, 0 - wf, sr = result - if wf.shape[0] > 1: wf = wf.mean(dim=0, keepdim=True) - return {"waveform": wf, "sample_rate": sr}, wf.shape[1] / sr - - def _trim(self, audio, td, st, et, sr, fps): - fd = 1.0 / fps - st = max(0.0, min(st, td)); et = max(st + fd, min(et, td)) - ss, es = int(st * sr), int(et * sr) - if es <= ss: es = min(ss + int(fd * 4 * sr), audio.shape[1]) - dur = (es - ss) / sr - raw = int(round(dur * fps)) - fc = _snap_4n1(raw) - sliced = audio[:, ss:es] - target = int((fc / fps) * sr) - if sliced.shape[1] < target: - pad = torch.zeros((sliced.shape[0], target - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) - sliced = torch.cat([sliced, pad], dim=1) - elif sliced.shape[1] > target: - sliced = sliced[:, :target] - return sliced, fc - - def _apply_mutes_v3(self, waveform, sample_rate, mute_data_str, start_offset): - """静音消音：必须减去 start_offset，将绝对时间转换为切片内的相对时间！""" - if not mute_data_str: return waveform - wav = waveform.clone() - for r in mute_data_str.split(';'): - if not r.strip(): continue - try: - s_str, e_str = r.split(',') - # 减去截取的起始时间，定位到当前波形张量中的真实物理位置 - rel_s = float(s_str) - start_offset - rel_e = float(e_str) - start_offset - s_idx = max(0, int(rel_s * sample_rate)) - e_idx = min(wav.shape[1], int(rel_e * sample_rate)) - if e_idx > s_idx: - wav[:, s_idx:e_idx] = 0.0 - except: pass - return wav - - def slice_v3(self, audio1, fps, start1, end1, mute_count1, mutes1_data, - audio2, start2, end2, mute_count2, mutes2_data, - merge_mode, total_display): - a1, d1f = self._load(audio1) - a2, d2f = self._load(audio2) - sr = 44100 - if a1: sr = a1["sample_rate"] - elif a2: sr = a2["sample_rate"] - - t1 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f1 = 0 - t2 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f2 = 0 - - if a1 and d1f > 0: - t1, f1 = self._trim(a1["waveform"], d1f, start1, end1, a1["sample_rate"], fps) - t1 = self._apply_mutes_v3(t1, a1["sample_rate"], mutes1_data, start1) - if a2 and d2f > 0: - t2, f2 = self._trim(a2["waveform"], d2f, start2, end2, a2["sample_rate"], fps) - t2 = self._apply_mutes_v3(t2, a2["sample_rate"], mutes2_data, start2) - - # 采样率对齐 - import torchaudio - if a2 and a2["sample_rate"] != sr: - t2 = torchaudio.functional.resample(t2, a2["sample_rate"], sr) - if a1 and a1["sample_rate"] != sr: - t1 = torchaudio.functional.resample(t1, a1["sample_rate"], sr) - - # 核心混音引擎 - if merge_mode == "接力": - combined = torch.cat([t1, t2], dim=1) - total_f = _snap_4n1(f1 + f2) - else: # 重叠混音模式 - len1, len2 = t1.shape[1], t2.shape[1] - max_len = max(len1, len2) - if len1 < max_len: - t1 = torch.cat([t1, torch.zeros((t1.shape[0], max_len - len1), dtype=t1.dtype, device=t1.device)], dim=1) - if len2 < max_len: - t2 = torch.cat([t2, torch.zeros((t2.shape[0], max_len - len2), dtype=t2.dtype, device=t2.device)], dim=1) - # 硬件级压限(Hard Limiter)，彻底斩断由于叠加造成的爆音 - combined = torch.clamp(t1 + t2, min=-1.0, max=1.0) - total_f = _snap_4n1(max(f1, f2)) - - # 最终长度保护 - target_samples = int((total_f / fps) * sr) - if combined.shape[1] < target_samples: - combined = torch.cat([combined, torch.zeros((combined.shape[0], target_samples - combined.shape[1]), dtype=combined.dtype, device=combined.device)], dim=1) - elif combined.shape[1] > target_samples: - combined = combined[:, :target_samples] - - return (_make_audio(combined, sr), total_f, fps, - _make_audio(t1, sr), f1, _make_audio(t2, sr), f2) +""" +XB-ToolBox 音频切片节点 +====================== +完全独立的音频加载/切片节点，支持可视化波形截取。 +使用 av (PyAV) 解码，兼容所有格式（同 VHS 用 ffmpeg 的效果） +""" + +import os, asyncio + +import folder_paths +import torch +import av +from aiohttp import web +import server + +from ...libs.xb_av import open_av + +def _load_audio_file(filepath: str): + """用 av 解码音频，返回 (waveform_2d, sample_rate)。加载失败返回 None""" + try: + with open_av(filepath) as af: + if not af.streams.audio: + return None + stream = af.streams.audio[0] + sr = stream.codec_context.sample_rate + frames = [] + for frame in af.decode(streams=stream.index): + buf = torch.from_numpy(frame.to_ndarray()) + if buf.dim() == 1: + buf = buf.unsqueeze(0) + elif buf.shape[0] != stream.channels: + buf = buf.view(-1, stream.channels).t() + frames.append(buf) + if not frames: + return None + wav = torch.cat(frames, dim=1) + # 统一转为 float32 + if not wav.dtype.is_floating_point: + if wav.dtype == torch.int16: + wav = wav.float() / (2 ** 15) + elif wav.dtype == torch.int32: + wav = wav.float() / (2 ** 31) + return wav, sr + except Exception: + return None + + +def _make_audio(waveform, sample_rate): + if waveform.dim() == 1: + waveform = waveform.unsqueeze(0).unsqueeze(0) + elif waveform.dim() == 2: + waveform = waveform.unsqueeze(0) + return {"waveform": waveform, "sample_rate": sample_rate} + +@server.PromptServer.instance.routes.post("/xb_audio_waveform") +@server.PromptServer.instance.routes.post("/xb_toolbox/audio_waveform") +async def handle_audio_waveform(request): + """返回音频文件的波形峰值数据，支持按时间范围切片重算""" + try: + data = await request.json() + except Exception: + return web.json_response({"error": "Invalid JSON body"}) + + filename = data.get("filename", "") + num_peaks = data.get("num_peaks", 400) + start_time = data.get("start_time", None) + end_time = data.get("end_time", None) + + if not filename: + return web.json_response({"peaks": [], "duration": 0}) + + input_dir = folder_paths.get_input_directory() + audio_path = os.path.join(input_dir, filename) + + if not os.path.exists(audio_path): + return web.json_response({"peaks": [], "duration": 0}) + + try: + loop = asyncio.get_event_loop() + result = await loop.run_in_executor(None, _load_audio_file, audio_path) + if result is None: + return web.json_response({"peaks": [], "duration": 0}) + waveform, sample_rate = result + if waveform.shape[0] > 1: + waveform = waveform.mean(dim=0, keepdim=True) + duration = waveform.shape[1] / sample_rate + + if start_time is not None and end_time is not None and end_time > start_time: + s_idx = max(0, int(start_time * sample_rate)) + e_idx = min(waveform.shape[1], int(end_time * sample_rate)) + waveform = waveform[:, s_idx:e_idx] + + total_samples = waveform.shape[1] + if total_samples == 0: + return web.json_response({"peaks": [], "duration": duration}) + + chunk_size = max(1, total_samples // num_peaks) + peaks = [] + for i in range(num_peaks): + chunk = waveform[0, i * chunk_size : (i + 1) * chunk_size] + if chunk.numel() == 0: + peaks.append([0.0, 0.0]) + continue + peaks.append([float(chunk.max().item()), float(chunk.min().item())]) + + return web.json_response({"peaks": peaks, "duration": duration}) + except Exception as e: + return web.json_response({"error": str(e)}) + + +def _snap_4n1(frames): + """将帧数对齐到 4N+1（满足 InfiniteTalk 的硬性要求）""" + return max(1, ((frames + 2) // 4) * 4 + 1) + + +# ============================================================ +# XB_AudioSlicer — 音频切片（基础版） +# ============================================================ +class XB_AudioSlicer: + """音频加载与切片节点 — 支持频谱可视化 + 拖拽分割线截取 + 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。 + 额外输出 fps，方便连接到总线统一帧率。""" + + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + audio_files = ["none"] + if os.path.exists(input_dir): + for f in sorted(os.listdir(input_dir)): + if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): + audio_files.append(f) + + return { + "required": { + "audio": (audio_files,), + "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), + "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01, "precision": 2, "round": 0.01}), + "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01, "precision": 2, "round": 0.01}), + "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), + } + } + + RETURN_TYPES = ("AUDIO", "INT", "FLOAT") + RETURN_NAMES = ("audio", "frame_count", "fps") + FUNCTION = "slice_audio" + CATEGORY = "XB_ToolBox/Audio" + + @classmethod + def VALIDATE_INPUTS(cls, audio, **kwargs): + return True + + def slice_audio(self, audio, fps, start_time, end_time, duration_display): + if audio == "none": + return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) + + result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), audio)) + if result is None: + return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) + + waveform, sample_rate = result + if waveform.shape[0] > 1: + waveform = waveform.mean(dim=0, keepdim=True) + + total_duration = waveform.shape[1] / sample_rate + frame_duration = 1.0 / fps + + start_time = max(0.0, min(start_time, total_duration)) + end_time = max(start_time + frame_duration, min(end_time, total_duration)) + + start_sample = int(start_time * sample_rate) + end_sample = int(end_time * sample_rate) + + if end_sample <= start_sample: + end_sample = min(start_sample + int(frame_duration * 4 * sample_rate), waveform.shape[1]) + + sliced = waveform[:, start_sample:end_sample] + duration_sec = (end_sample - start_sample) / sample_rate + raw_frames = int(round(duration_sec * fps)) + frame_count = _snap_4n1(raw_frames) + # 🛡️ 音频物理长度对齐 4N+1 帧数，防止音视频时间轴错位 + target_samples = int((frame_count / fps) * sample_rate) + if sliced.shape[1] < target_samples: + pad = torch.zeros((sliced.shape[0], target_samples - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) + sliced = torch.cat([sliced, pad], dim=1) + elif sliced.shape[1] > target_samples: + sliced = sliced[:, :target_samples] + if frame_count != raw_frames: + print(f"🔧 [音频切片] 帧数 {raw_frames} → {frame_count} (对齐 4N+1)") + return (_make_audio(sliced, sample_rate), frame_count, fps) + + +# ============================================================ +# XB_AudioSlicerV1 — 可视化音频切片（波形+拖拽分割线） +# ============================================================ +class XB_AudioSlicerV1: + """音频切片节点 V1 — 可视化波形窗口 + 拖拽分割线截取 + 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。 + 额外输出 fps，方便连接到总线统一帧率。""" + + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + audio_files = ["none"] + if os.path.exists(input_dir): + for f in sorted(os.listdir(input_dir)): + if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): + audio_files.append(f) + + return { + "required": { + "audio": (audio_files,), + "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), + "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), + } + } + + RETURN_TYPES = ("AUDIO", "INT", "FLOAT") + RETURN_NAMES = ("audio", "frame_count", "fps") + FUNCTION = "slice_audio" + CATEGORY = "XB_ToolBox/Audio" + + @classmethod + def VALIDATE_INPUTS(cls, audio, **kwargs): + return True + + def slice_audio(self, audio, fps, start_time, end_time, duration_display): + if audio == "none": + return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) + + result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), audio)) + if result is None: + return (_make_audio(torch.zeros((1, 1), dtype=torch.float32), 44100), 0, fps) + + waveform, sample_rate = result + if waveform.shape[0] > 1: + waveform = waveform.mean(dim=0, keepdim=True) + + total_duration = waveform.shape[1] / sample_rate + frame_duration = 1.0 / fps + start_time = max(0.0, min(start_time, total_duration)) + end_time = max(start_time + frame_duration, min(end_time, total_duration)) + + start_sample = int(start_time * sample_rate) + end_sample = int(end_time * sample_rate) + if end_sample <= start_sample: + end_sample = min(start_sample + int(frame_duration * 4 * sample_rate), waveform.shape[1]) + + sliced = waveform[:, start_sample:end_sample] + duration_sec = (end_sample - start_sample) / sample_rate + raw_frames = int(round(duration_sec * fps)) + frame_count = _snap_4n1(raw_frames) + if frame_count != raw_frames: + print(f"🔧 [音频切片V1] 帧数 {raw_frames} → {frame_count} (对齐 4N+1)") + return (_make_audio(sliced, sample_rate), frame_count, fps) + + +# ============================================================ +# XB_AudioSlicerV2 — 双人音频切片 +# ============================================================ +class XB_AudioSlicerV2: + """双人音频切片 — 音频1 + 间隔(帧) + 音频2，独立可视化 + 输出帧数（非秒数），可直接对接 InfiniteTalk 接力点的 segment_length。""" + + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + audio_files = ["none"] + if os.path.exists(input_dir): + for f in sorted(os.listdir(input_dir)): + if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): + audio_files.append(f) + + return { + "required": { + "audio1": (audio_files,), + "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), + "start1": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "end1": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "audio2": (audio_files,), + "start2": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "end2": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "gap_frames": ("INT", {"default": 25, "min": 0, "max": 99999, "step": 1}), + "total_display": ("STRING", {"default": "0 帧", "multiline": False}), + } + } + + RETURN_TYPES = ("AUDIO", "INT", "FLOAT", "AUDIO", "INT", "AUDIO", "INT") + RETURN_NAMES = ("combined_audio", "total_frames", "fps", "audio1", "frames1", "audio2", "frames2") + FUNCTION = "slice_dual" + CATEGORY = "XB_ToolBox/Audio" + + @classmethod + def VALIDATE_INPUTS(cls, audio1, audio2, **kwargs): + return True + + def _load(self, name): + if name == "none": return None, 0 + result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), name)) + if result is None: return None, 0 + wf, sr = result + if wf.shape[0] > 1: wf = wf.mean(dim=0, keepdim=True) + return {"waveform": wf, "sample_rate": sr}, wf.shape[1] / sr + + def _trim(self, audio, td, st, et, sr, fps): + fd = 1.0 / fps + st = max(0.0, min(st, td)); et = max(st + fd, min(et, td)) + ss, es = int(st * sr), int(et * sr) + if es <= ss: es = min(ss + int(fd * 4 * sr), audio.shape[1]) + dur = (es - ss) / sr + raw = int(round(dur * fps)) + fc = _snap_4n1(raw) + # 🛡️ 音频物理长度对齐 4N+1 帧数 + sliced = audio[:, ss:es] + target = int((fc / fps) * sr) + if sliced.shape[1] < target: + pad = torch.zeros((sliced.shape[0], target - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) + sliced = torch.cat([sliced, pad], dim=1) + elif sliced.shape[1] > target: + sliced = sliced[:, :target] + return sliced, fc + + def slice_dual(self, audio1, fps, start1, end1, audio2, start2, end2, gap_frames, total_display): + a1, d1f = self._load(audio1); a2, d2f = self._load(audio2) + sr = 44100 + if a1: sr = a1["sample_rate"] + elif a2: sr = a2["sample_rate"] + + # 各自用原生采样率裁剪 + t1 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f1 = 0 + t2 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f2 = 0 + if a1 and d1f > 0: t1, f1 = self._trim(a1["waveform"], d1f, start1, end1, a1["sample_rate"], fps) + if a2 and d2f > 0: t2, f2 = self._trim(a2["waveform"], d2f, start2, end2, a2["sample_rate"], fps) + + # 统一采样率 + import torchaudio + if a2 and a2["sample_rate"] != sr: + t2 = torchaudio.functional.resample(t2, a2["sample_rate"], sr) + if a1 and a1["sample_rate"] != sr: + t1 = torchaudio.functional.resample(t1, a1["sample_rate"], sr) + + gap_s = int(max(0, gap_frames) / fps * sr) + gap_wf = torch.zeros((1, gap_s), dtype=torch.float32) + combined = torch.cat([t1, gap_wf, t2], dim=1) + total_f = _snap_4n1(f1 + max(0, gap_frames) + f2) + if total_f != f1 + max(0, gap_frames) + f2: + print(f"🔧 [音频切片V2] 合并帧数 {f1 + max(0, gap_frames) + f2} → {total_f} (对齐 4N+1)") + return (_make_audio(combined, sr), total_f, fps, + _make_audio(t1, sr), f1, _make_audio(t2, sr), f2) + + +# ============================================================ +# XB_AudioSlicerV3 — 高级双轨混音与静音切片 (视窗缩放版) +# ============================================================ +class XB_AudioSlicerV3: + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + audio_files = ["none"] + if os.path.exists(input_dir): + for f in sorted(os.listdir(input_dir)): + if f.lower().endswith(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac', '.opus', '.wma', '.webm')): + audio_files.append(f) + + return { + "required": { + "audio1": (audio_files,), + "fps": ("FLOAT", {"default": 24.0, "min": 1.0, "max": 120.0, "step": 1.0}), + "start1": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 3600.0, "step": 0.01}), + "end1": ("FLOAT", {"default": 10.0, "min": 0.0, "max": 3600.0, "step": 0.01}), + "mute_count1": ("INT", {"default": 0, "min": 0, "max": 20, "step": 1}), + "mutes1_data": ("STRING", {"default": ""}), + "audio2": (audio_files,), + "start2": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 3600.0, "step": 0.01}), + "end2": ("FLOAT", {"default": 10.0, "min": 0.0, "max": 3600.0, "step": 0.01}), + "mute_count2": ("INT", {"default": 0, "min": 0, "max": 20, "step": 1}), + "mutes2_data": ("STRING", {"default": ""}), + "merge_mode": (["接力", "重叠"], {"default": "接力"}), + "total_display": ("STRING", {"default": "0 / 0帧", "multiline": False}), + } + } + + RETURN_TYPES = ("AUDIO", "INT", "FLOAT", "AUDIO", "INT", "AUDIO", "INT") + RETURN_NAMES = ("combined_audio", "total_frames", "fps", "audio1", "frames1", "audio2", "frames2") + FUNCTION = "slice_v3" + CATEGORY = "XB_ToolBox/Audio" + + @classmethod + def VALIDATE_INPUTS(cls, audio1, audio2, **kwargs): + return True + + def _load(self, name): + if name == "none": return None, 0 + result = _load_audio_file(os.path.join(folder_paths.get_input_directory(), name)) + if result is None: return None, 0 + wf, sr = result + if wf.shape[0] > 1: wf = wf.mean(dim=0, keepdim=True) + return {"waveform": wf, "sample_rate": sr}, wf.shape[1] / sr + + def _trim(self, audio, td, st, et, sr, fps): + fd = 1.0 / fps + st = max(0.0, min(st, td)); et = max(st + fd, min(et, td)) + ss, es = int(st * sr), int(et * sr) + if es <= ss: es = min(ss + int(fd * 4 * sr), audio.shape[1]) + dur = (es - ss) / sr + raw = int(round(dur * fps)) + fc = _snap_4n1(raw) + sliced = audio[:, ss:es] + target = int((fc / fps) * sr) + if sliced.shape[1] < target: + pad = torch.zeros((sliced.shape[0], target - sliced.shape[1]), dtype=sliced.dtype, device=sliced.device) + sliced = torch.cat([sliced, pad], dim=1) + elif sliced.shape[1] > target: + sliced = sliced[:, :target] + return sliced, fc + + def _apply_mutes_v3(self, waveform, sample_rate, mute_data_str, start_offset): + """静音消音：必须减去 start_offset，将绝对时间转换为切片内的相对时间！""" + if not mute_data_str: return waveform + wav = waveform.clone() + for r in mute_data_str.split(';'): + if not r.strip(): continue + try: + s_str, e_str = r.split(',') + # 减去截取的起始时间，定位到当前波形张量中的真实物理位置 + rel_s = float(s_str) - start_offset + rel_e = float(e_str) - start_offset + s_idx = max(0, int(rel_s * sample_rate)) + e_idx = min(wav.shape[1], int(rel_e * sample_rate)) + if e_idx > s_idx: + wav[:, s_idx:e_idx] = 0.0 + except: pass + return wav + + def slice_v3(self, audio1, fps, start1, end1, mute_count1, mutes1_data, + audio2, start2, end2, mute_count2, mutes2_data, + merge_mode, total_display): + a1, d1f = self._load(audio1) + a2, d2f = self._load(audio2) + sr = 44100 + if a1: sr = a1["sample_rate"] + elif a2: sr = a2["sample_rate"] + + t1 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f1 = 0 + t2 = torch.zeros((1, int(0.04 * sr)), dtype=torch.float32); f2 = 0 + + if a1 and d1f > 0: + t1, f1 = self._trim(a1["waveform"], d1f, start1, end1, a1["sample_rate"], fps) + t1 = self._apply_mutes_v3(t1, a1["sample_rate"], mutes1_data, start1) + if a2 and d2f > 0: + t2, f2 = self._trim(a2["waveform"], d2f, start2, end2, a2["sample_rate"], fps) + t2 = self._apply_mutes_v3(t2, a2["sample_rate"], mutes2_data, start2) + + # 采样率对齐 + import torchaudio + if a2 and a2["sample_rate"] != sr: + t2 = torchaudio.functional.resample(t2, a2["sample_rate"], sr) + if a1 and a1["sample_rate"] != sr: + t1 = torchaudio.functional.resample(t1, a1["sample_rate"], sr) + + # 核心混音引擎 + if merge_mode == "接力": + combined = torch.cat([t1, t2], dim=1) + total_f = _snap_4n1(f1 + f2) + else: # 重叠混音模式 + len1, len2 = t1.shape[1], t2.shape[1] + max_len = max(len1, len2) + if len1 < max_len: + t1 = torch.cat([t1, torch.zeros((t1.shape[0], max_len - len1), dtype=t1.dtype, device=t1.device)], dim=1) + if len2 < max_len: + t2 = torch.cat([t2, torch.zeros((t2.shape[0], max_len - len2), dtype=t2.dtype, device=t2.device)], dim=1) + # 硬件级压限(Hard Limiter)，彻底斩断由于叠加造成的爆音 + combined = torch.clamp(t1 + t2, min=-1.0, max=1.0) + total_f = _snap_4n1(max(f1, f2)) + + # 最终长度保护 + target_samples = int((total_f / fps) * sr) + if combined.shape[1] < target_samples: + combined = torch.cat([combined, torch.zeros((combined.shape[0], target_samples - combined.shape[1]), dtype=combined.dtype, device=combined.device)], dim=1) + elif combined.shape[1] > target_samples: + combined = combined[:, :target_samples] + + return (_make_audio(combined, sr), total_f, fps, + _make_audio(t1, sr), f1, _make_audio(t2, sr), f2) diff --git a/nodes_play_sound.py b/nodes/audio/nodes_play_sound.py similarity index 100% rename from nodes_play_sound.py rename to nodes/audio/nodes_play_sound.py diff --git a/nodes_yue2_lyrics.py b/nodes/audio/nodes_yue2_lyrics.py similarity index 99% rename from nodes_yue2_lyrics.py rename to nodes/audio/nodes_yue2_lyrics.py index 066d8f5..9b7d8a1 100644 --- a/nodes_yue2_lyrics.py +++ b/nodes/audio/nodes_yue2_lyrics.py @@ -22,7 +22,7 @@ CATEGORY: JZL/YuE2 import re try: - from .yue2_abc import OFFICIAL_SECTION_TAGS + from ...libs.yue2_abc import OFFICIAL_SECTION_TAGS except ImportError: # 以文件方式直接加载时的兜底 from yue2_abc import OFFICIAL_SECTION_TAGS diff --git a/nodes_yue2_score.py b/nodes/audio/nodes_yue2_score.py similarity index 99% rename from nodes_yue2_score.py rename to nodes/audio/nodes_yue2_score.py index 82f0a26..b2a0d20 100644 --- a/nodes_yue2_score.py +++ b/nodes/audio/nodes_yue2_score.py @@ -26,7 +26,7 @@ CATEGORY: JZL/YuE2 import json try: - from .yue2_abc import ( + from ...libs.yue2_abc import ( AbcError, TOKEN, VOICES, compare_text, compile_events, convert_score, diff_lines, mode_suggestion, parse_abc, sha_line, strip_chords, summary_text, validate_instrumental, diff --git a/nodes_yue2_style.py b/nodes/audio/nodes_yue2_style.py similarity index 99% rename from nodes_yue2_style.py rename to nodes/audio/nodes_yue2_style.py index 23be441..20484b6 100644 --- a/nodes_yue2_style.py +++ b/nodes/audio/nodes_yue2_style.py @@ -20,7 +20,7 @@ CATEGORY: JZL/YuE2 import re try: - from .yue2_styles import ( + from ...libs.yue2_styles import ( BPM_HINTS, DENSITY_HINTS, GENRES, INSTRUMENT_HINTS, LANGUAGE_HINTS, MOOD_HINTS, RECIPES, TECHNIQUE_HINTS, TEXTURE_HINTS, TIMBRE_HINTS, VOCAL_FX_HINTS, VOCAL_HINTS, FOLLOW, CUSTOM, genre_options, diff --git a/nodes_yue2_transcription.py b/nodes/audio/nodes_yue2_transcription.py similarity index 99% rename from nodes_yue2_transcription.py rename to nodes/audio/nodes_yue2_transcription.py index 32d95f4..641e706 100644 --- a/nodes_yue2_transcription.py +++ b/nodes/audio/nodes_yue2_transcription.py @@ -22,7 +22,7 @@ import os import pathlib try: - from .yue2_abc import ( + from ...libs.yue2_abc import ( AbcError, compile_events, diff_lines, mode_suggestion, normalize_abc_text, parse_abc, sha_line, summary_text, ) diff --git a/cosyvoice3/cosyvoice/utils/__init__.py b/nodes/core/__init__.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/__init__.py rename to nodes/core/__init__.py diff --git a/nodes_batch.py b/nodes/core/nodes_batch.py similarity index 97% rename from nodes_batch.py rename to nodes/core/nodes_batch.py index cf124f2..79d5f5e 100644 --- a/nodes_batch.py +++ b/nodes/core/nodes_batch.py @@ -1,128 +1,128 @@ -import os -import torch -import numpy as np -import re -from PIL import Image, ImageOps -import folder_paths -import datetime - -from .nodes_batch_images import batch_images - -VALID_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.webp', '.bmp', '.jfif'} - - -def _natural_sort_key(s): - return [int(text) if text.isdigit() else text.lower() for text in re.split('([0-9]+)', s)] - - -def _list_image_files(directory): - if not os.path.isdir(directory): - raise FileNotFoundError(f"Directory not found: {directory}") - files = [f for f in os.listdir(directory) if os.path.splitext(f)[1].lower() in VALID_EXTENSIONS] - files.sort(key=_natural_sort_key) - return files - - -def _load_image_tensor(file_path): - """读取单张图片 → (1, H, W, C) float32 张量。""" - img = Image.open(file_path) - img = ImageOps.exif_transpose(img) - image = img.convert("RGB") - image = np.array(image).astype(np.float32) / 255.0 - return torch.from_numpy(image)[None,] - - -# ============================================================ -# XB_BatchFolderLoader — 图片序列加载（单张步进） -# ============================================================ -class XB_BatchFolderLoader: - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "directory": ("STRING", {"default": ""}), - "seed": ("INT", {"default": 0, "min": 0, "max": 9999999, "step": 1, "tooltip": "Built-in auto-stepper using the seed mechanism"}), - } - } - - RETURN_TYPES = ("IMAGE", "MASK", "STRING", "STRING") - RETURN_NAMES = ("IMAGE", "MASK", "FILE_NAME", "TOTAL_COUNT") - FUNCTION = "load_images" - CATEGORY = "XB_ToolBox/Batch_Process" - - def load_images(self, directory, seed): - if not os.path.isdir(directory): - raise FileNotFoundError(f"Directory not found: {directory}") - - valid_extensions = {'.png', '.jpg', '.jpeg', '.webp', '.bmp', '.jfif'} - files = [f for f in os.listdir(directory) if os.path.splitext(f)[1].lower() in valid_extensions] - - def natural_sort_key(s): - return [int(text) if text.isdigit() else text.lower() for text in re.split('([0-9]+)', s)] - - files.sort(key=natural_sort_key) - - if not files: - raise Exception(f"No valid images found in directory: {directory}") - - total_count = len(files) - current_file_idx = seed % total_count - file_path = os.path.join(directory, files[current_file_idx]) - - raw_name = os.path.splitext(files[current_file_idx])[0] - today_str = datetime.datetime.now().strftime("%Y-%m-%d") - formatted_file_name = f"Images/Batch/{today_str}/{raw_name}" - total_count_display = f"Total images detected: {total_count}" - - img = Image.open(file_path) - img = ImageOps.exif_transpose(img) - image = img.convert("RGB") - image = np.array(image).astype(np.float32) / 255.0 - image = torch.from_numpy(image)[None,] - - if 'A' in img.getbands(): - mask = np.array(img.getchannel('A')).astype(np.float32) / 255.0 - mask = 1. - torch.from_numpy(mask) - mask = mask.unsqueeze(0) # (H, W) → (1, H, W) - else: - mask = torch.zeros((1, image.shape[1], image.shape[2]), dtype=torch.float32) - - return (image, mask, formatted_file_name, total_count_display) - - -# ============================================================ -# XB_BatchFolderBatchLoader — 图片批量加载（一次读取多张） -# ============================================================ -class XB_BatchFolderBatchLoader: - """选择文件夹后一次性批量加载多张图片，合并成一个图像批次传给下游（等价于批量图像节点接多张图）。""" - - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "directory": ("STRING", {"default": ""}), - "count": ("INT", {"default": 4, "min": 1, "max": 500, "step": 1, - "tooltip": "本次批量加载的图片数量"}), - "skip": ("INT", {"default": 0, "min": 0, "max": 999999, "step": 1, - "tooltip": "跳过前 N 张，便于多次分批处理"}), - } - } - - RETURN_TYPES = ("IMAGE",) - RETURN_NAMES = ("IMAGES",) - FUNCTION = "load_batch" - CATEGORY = "XB_ToolBox/Batch_Process" - - def load_batch(self, directory, count, skip): - files = _list_image_files(directory) - if not files: - raise Exception(f"No valid images found in directory: {directory}") - - selected = files[skip:skip + count] - if not selected: - raise Exception( - f"No images left: skipped {skip} of {len(files)} images in {directory}" - ) - - images = [_load_image_tensor(os.path.join(directory, f)) for f in selected] +import os +import torch +import numpy as np +import re +from PIL import Image, ImageOps +import folder_paths +import datetime + +from .nodes_batch_images import batch_images + +VALID_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.webp', '.bmp', '.jfif'} + + +def _natural_sort_key(s): + return [int(text) if text.isdigit() else text.lower() for text in re.split('([0-9]+)', s)] + + +def _list_image_files(directory): + if not os.path.isdir(directory): + raise FileNotFoundError(f"Directory not found: {directory}") + files = [f for f in os.listdir(directory) if os.path.splitext(f)[1].lower() in VALID_EXTENSIONS] + files.sort(key=_natural_sort_key) + return files + + +def _load_image_tensor(file_path): + """读取单张图片 → (1, H, W, C) float32 张量。""" + img = Image.open(file_path) + img = ImageOps.exif_transpose(img) + image = img.convert("RGB") + image = np.array(image).astype(np.float32) / 255.0 + return torch.from_numpy(image)[None,] + + +# ============================================================ +# XB_BatchFolderLoader — 图片序列加载（单张步进） +# ============================================================ +class XB_BatchFolderLoader: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "directory": ("STRING", {"default": ""}), + "seed": ("INT", {"default": 0, "min": 0, "max": 9999999, "step": 1, "tooltip": "Built-in auto-stepper using the seed mechanism"}), + } + } + + RETURN_TYPES = ("IMAGE", "MASK", "STRING", "STRING") + RETURN_NAMES = ("IMAGE", "MASK", "FILE_NAME", "TOTAL_COUNT") + FUNCTION = "load_images" + CATEGORY = "XB_ToolBox/Batch_Process" + + def load_images(self, directory, seed): + if not os.path.isdir(directory): + raise FileNotFoundError(f"Directory not found: {directory}") + + valid_extensions = {'.png', '.jpg', '.jpeg', '.webp', '.bmp', '.jfif'} + files = [f for f in os.listdir(directory) if os.path.splitext(f)[1].lower() in valid_extensions] + + def natural_sort_key(s): + return [int(text) if text.isdigit() else text.lower() for text in re.split('([0-9]+)', s)] + + files.sort(key=natural_sort_key) + + if not files: + raise Exception(f"No valid images found in directory: {directory}") + + total_count = len(files) + current_file_idx = seed % total_count + file_path = os.path.join(directory, files[current_file_idx]) + + raw_name = os.path.splitext(files[current_file_idx])[0] + today_str = datetime.datetime.now().strftime("%Y-%m-%d") + formatted_file_name = f"Images/Batch/{today_str}/{raw_name}" + total_count_display = f"Total images detected: {total_count}" + + img = Image.open(file_path) + img = ImageOps.exif_transpose(img) + image = img.convert("RGB") + image = np.array(image).astype(np.float32) / 255.0 + image = torch.from_numpy(image)[None,] + + if 'A' in img.getbands(): + mask = np.array(img.getchannel('A')).astype(np.float32) / 255.0 + mask = 1. - torch.from_numpy(mask) + mask = mask.unsqueeze(0) # (H, W) → (1, H, W) + else: + mask = torch.zeros((1, image.shape[1], image.shape[2]), dtype=torch.float32) + + return (image, mask, formatted_file_name, total_count_display) + + +# ============================================================ +# XB_BatchFolderBatchLoader — 图片批量加载（一次读取多张） +# ============================================================ +class XB_BatchFolderBatchLoader: + """选择文件夹后一次性批量加载多张图片，合并成一个图像批次传给下游（等价于批量图像节点接多张图）。""" + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "directory": ("STRING", {"default": ""}), + "count": ("INT", {"default": 4, "min": 1, "max": 500, "step": 1, + "tooltip": "本次批量加载的图片数量"}), + "skip": ("INT", {"default": 0, "min": 0, "max": 999999, "step": 1, + "tooltip": "跳过前 N 张，便于多次分批处理"}), + } + } + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("IMAGES",) + FUNCTION = "load_batch" + CATEGORY = "XB_ToolBox/Batch_Process" + + def load_batch(self, directory, count, skip): + files = _list_image_files(directory) + if not files: + raise Exception(f"No valid images found in directory: {directory}") + + selected = files[skip:skip + count] + if not selected: + raise Exception( + f"No images left: skipped {skip} of {len(files)} images in {directory}" + ) + + images = [_load_image_tensor(os.path.join(directory, f)) for f in selected] return (batch_images(images),) \ No newline at end of file diff --git a/nodes_batch_images.py b/nodes/core/nodes_batch_images.py similarity index 100% rename from nodes_batch_images.py rename to nodes/core/nodes_batch_images.py diff --git a/nodes_blockswap.py b/nodes/core/nodes_blockswap.py similarity index 97% rename from nodes_blockswap.py rename to nodes/core/nodes_blockswap.py index d30ee3f..202e4c1 100644 --- a/nodes_blockswap.py +++ b/nodes/core/nodes_blockswap.py @@ -1,232 +1,232 @@ -import comfy.model_management as mm -import gc -import torch -import sys -from comfy.patcher_extension import CallbacksMP -from comfy.model_patcher import ModelPatcher - -try: - from tqdm import tqdm -except ImportError: - def tqdm(iterable, **kwargs): - return iterable - -class AnyType(str): - def __ne__(self, __value: object) -> bool: - return False -anyType = AnyType("*") - -def is_dynamic_vram_active(): - """仅当用户显式传参 --enable-dynamic-vram 时视为动态显存激活，分块节点休眠。""" - if "--enable-dynamic-vram" in sys.argv: - print("\n\033[93m[XB Block Swap]\033[0m: ⚠️ 检测到 --enable-dynamic-vram，动态显存已启用。分块节点自动休眠。") - return True - return False - -def is_unsupported_model(diffusion_model): - """检测已知不支持物理分块的模型架构（仅黑名单，不检测量化格式）。 - - Qwen / FLUX 等模型的 GGUF 版本虽然参数类型特殊，但 module.to() 仍可正常工作， - 不应被拦截。只有当模型的 C++ 量化引擎（如 comfy_kitchen）确实会因 module.to() - 而崩溃时才需要拦截——但这类崩溃无法通过参数类型静态判断，由用户自行取舍。 - """ - model_type = type(diffusion_model).__name__ - blacklist = ["Lumina", "Lumina2", "ZImage", "HunyuanDiT", "ErnieImageModel"] - for b in blacklist: - if b in model_type: - return model_type - return None - - -# ============================================================ -# XB_UNetBlockSwap — UNet 分块交换 (显存优化) -# ============================================================ -class XB_UNetBlockSwap: - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "unet_model": (anyType, {"tooltip": "Accepts pure UNet/DiT model input only"}), - "blocks_to_swap": ("INT", { - "default": 10, - "min": 0, - "max": 200, - "step": 1, - "tooltip": "Number of core blocks to offload to system RAM. Higher values save VRAM but slow down generation." - }), - }, - } - - RETURN_TYPES = (anyType,) - RETURN_NAMES = ("unet_model",) - CATEGORY = "XB_ToolBox/VRAM_Hacks" - FUNCTION = "set_callback" - - def set_callback(self, unet_model: ModelPatcher, blocks_to_swap): - if not isinstance(unet_model, ModelPatcher): - return (unet_model,) - - # ── 0 = 关闭分块功能：直接穿透，不做任何检测、不输出动态显存休眠警告 ── - if blocks_to_swap == 0: - print("\033[96m[XB UNet Block Swap]\033[0m: 分块数量 = 0，分块功能已关闭（节点穿透）") - return (unet_model,) - - if is_dynamic_vram_active(): - return (unet_model,) - - def swap_blocks(model_patcher: ModelPatcher, device_to, lowvram_model_memory, force_patch_weights, full_load): - base_model = model_patcher.model - # 🛡️ 动态获取 ComfyUI 分配的 GPU 设备，绝不写死 'cuda:0' - main_device = model_patcher.load_device - - diffusion_model = getattr(base_model, 'diffusion_model', None) - if not diffusion_model: - return - - unsupported_name = is_unsupported_model(diffusion_model) - if unsupported_name: - print(f"\033[93m[XB-BOX 拦截盾]\033[0m: 模型 ({unsupported_name}) 属于高耦合架构，不支持物理分块。已自动跳过！") - return - - all_blocks = [] - block_paths = [ - 'transformer_blocks', 'blocks', 'down_blocks', 'up_blocks', 'mid_block', - 'layers', 'attention_blocks', 'input_blocks', 'middle_block', 'output_blocks', - 'double_blocks', 'single_blocks', 'joint_blocks' - ] - - for path in block_paths: - if hasattr(diffusion_model, path): - attr = getattr(diffusion_model, path) - if isinstance(attr, (list, torch.nn.ModuleList)): - for item in attr: - all_blocks.append(item) - elif isinstance(attr, torch.nn.Module) and path in ['mid_block', 'middle_block']: - all_blocks.append(attr) - - if not all_blocks: - return - - total_blocks = len(all_blocks) - # 🛡️ 防呆：用户输入不可超过模型总块数 - if blocks_to_swap > total_blocks: - effective = total_blocks - tag = "匹配最大可分块参数" - else: - effective = blocks_to_swap - tag = "匹配用户设置的参数" - print(f"\033[96m[XB UNet Block Swap]\033[0m: 静态物理分块交换已激活！已找到 {total_blocks} 个引擎模块，分割 {effective} 个分块（{tag}）。") - - for b, block in tqdm(enumerate(all_blocks), total=total_blocks, desc="Slicing UNet pipeline"): - if b > effective: - block.to(main_device) - else: - block.to(model_patcher.offload_device) - - mm.soft_empty_cache() - gc.collect() - - unet_model = unet_model.clone() - unet_model.add_callback(CallbacksMP.ON_LOAD, swap_blocks) - - return (unet_model, ) - -# ============================================================ -# XB_CheckpointBlockSwap — Checkpoint 分块交换 (显存优化) -# ============================================================ -class XB_CheckpointBlockSwap: - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "checkpoint_model": (anyType, {"tooltip": "Accepts any model input (including CLIP/VAE)."}), - "blocks_to_swap": ("INT", {"default": 15, "min": 0, "max": 1000, "step": 1}), - "offload_img_emb": ("BOOLEAN", {"default": True}), - "offload_txt_emb": ("BOOLEAN", {"default": True}), - "use_non_blocking": ("BOOLEAN", {"default": False}), - }, - } - - RETURN_TYPES = (anyType,) - RETURN_NAMES = ("checkpoint_model",) - CATEGORY = "XB_ToolBox/VRAM_Hacks" - FUNCTION = "set_callback" - - def set_callback(self, checkpoint_model: ModelPatcher, blocks_to_swap, offload_txt_emb, offload_img_emb, use_non_blocking): - if not isinstance(checkpoint_model, ModelPatcher): - return (checkpoint_model,) - - # ── 0 = 关闭分块功能：直接穿透，不做任何检测、不输出动态显存休眠警告 ── - if blocks_to_swap == 0: - print("\033[96m[XB Checkpoint Block Swap]\033[0m: 分块数量 = 0，分块功能已关闭（节点穿透）") - return (checkpoint_model,) - - if is_dynamic_vram_active(): - return (checkpoint_model,) - - def swap_blocks(model_patcher: ModelPatcher, device_to, lowvram_model_memory, force_patch_weights, full_load): - base_model = model_patcher.model - # 🛡️ 动态获取 ComfyUI 分配的 GPU 设备 - main_device = model_patcher.load_device - - diffusion_model = getattr(base_model, 'diffusion_model', None) - if not diffusion_model: - return - - unsupported_name = is_unsupported_model(diffusion_model) - if unsupported_name: - print(f"\033[93m[XB-BOX 拦截盾]\033[0m: 模型 ({unsupported_name}) 属于高耦合架构，不支持物理分块。已自动跳过！") - return - - all_blocks = [] - block_paths = [ - 'transformer_blocks', 'blocks', 'down_blocks', 'up_blocks', 'mid_block', - 'layers', 'attention_blocks', 'input_blocks', 'middle_block', 'output_blocks', - 'double_blocks', 'single_blocks', 'joint_blocks' - ] - - for path in block_paths: - if hasattr(diffusion_model, path): - attr = getattr(diffusion_model, path) - if isinstance(attr, (list, torch.nn.ModuleList)): - for item in attr: - all_blocks.append(item) - elif isinstance(attr, torch.nn.Module) and path in ['mid_block', 'middle_block']: - all_blocks.append(attr) - - if all_blocks: - total_blocks = len(all_blocks) - # 🛡️ 防呆：用户输入不可超过模型总块数 - if blocks_to_swap > total_blocks: - effective = total_blocks - tag = "匹配最大可分块参数" - else: - effective = blocks_to_swap - tag = "匹配用户设置的参数" - print(f"\033[96m[XB Checkpoint Block Swap]\033[0m: 静态物理分块交换已激活！已找到 {total_blocks} 个引擎模块，分割 {effective} 个分块（{tag}）。") - - for b, block in tqdm(enumerate(all_blocks), total=total_blocks, desc="Slicing Checkpoint pipeline"): - if b > effective: - block.to(main_device) - else: - block.to(model_patcher.offload_device) - - if offload_txt_emb: - for path in ['text_embedding', 'caption_encoder', 'text_encoder']: - if hasattr(diffusion_model, path) and getattr(diffusion_model, path) is not None: - getattr(diffusion_model, path).to(model_patcher.offload_device, non_blocking=use_non_blocking) - break - - if offload_img_emb: - for path in ['img_emb', 'image_encoder', 'visual_encoder']: - if hasattr(diffusion_model, path) and getattr(diffusion_model, path) is not None: - getattr(diffusion_model, path).to(model_patcher.offload_device, non_blocking=use_non_blocking) - break - - mm.soft_empty_cache() - gc.collect() - - checkpoint_model = checkpoint_model.clone() - checkpoint_model.add_callback(CallbacksMP.ON_LOAD, swap_blocks) - +import comfy.model_management as mm +import gc +import torch +import sys +from comfy.patcher_extension import CallbacksMP +from comfy.model_patcher import ModelPatcher + +try: + from tqdm import tqdm +except ImportError: + def tqdm(iterable, **kwargs): + return iterable + +class AnyType(str): + def __ne__(self, __value: object) -> bool: + return False +anyType = AnyType("*") + +def is_dynamic_vram_active(): + """仅当用户显式传参 --enable-dynamic-vram 时视为动态显存激活，分块节点休眠。""" + if "--enable-dynamic-vram" in sys.argv: + print("\n\033[93m[XB Block Swap]\033[0m: ⚠️ 检测到 --enable-dynamic-vram，动态显存已启用。分块节点自动休眠。") + return True + return False + +def is_unsupported_model(diffusion_model): + """检测已知不支持物理分块的模型架构（仅黑名单，不检测量化格式）。 + + Qwen / FLUX 等模型的 GGUF 版本虽然参数类型特殊，但 module.to() 仍可正常工作， + 不应被拦截。只有当模型的 C++ 量化引擎（如 comfy_kitchen）确实会因 module.to() + 而崩溃时才需要拦截——但这类崩溃无法通过参数类型静态判断，由用户自行取舍。 + """ + model_type = type(diffusion_model).__name__ + blacklist = ["Lumina", "Lumina2", "ZImage", "HunyuanDiT", "ErnieImageModel"] + for b in blacklist: + if b in model_type: + return model_type + return None + + +# ============================================================ +# XB_UNetBlockSwap — UNet 分块交换 (显存优化) +# ============================================================ +class XB_UNetBlockSwap: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "unet_model": (anyType, {"tooltip": "Accepts pure UNet/DiT model input only"}), + "blocks_to_swap": ("INT", { + "default": 10, + "min": 0, + "max": 200, + "step": 1, + "tooltip": "Number of core blocks to offload to system RAM. Higher values save VRAM but slow down generation." + }), + }, + } + + RETURN_TYPES = (anyType,) + RETURN_NAMES = ("unet_model",) + CATEGORY = "XB_ToolBox/VRAM_Hacks" + FUNCTION = "set_callback" + + def set_callback(self, unet_model: ModelPatcher, blocks_to_swap): + if not isinstance(unet_model, ModelPatcher): + return (unet_model,) + + # ── 0 = 关闭分块功能：直接穿透，不做任何检测、不输出动态显存休眠警告 ── + if blocks_to_swap == 0: + print("\033[96m[XB UNet Block Swap]\033[0m: 分块数量 = 0，分块功能已关闭（节点穿透）") + return (unet_model,) + + if is_dynamic_vram_active(): + return (unet_model,) + + def swap_blocks(model_patcher: ModelPatcher, device_to, lowvram_model_memory, force_patch_weights, full_load): + base_model = model_patcher.model + # 🛡️ 动态获取 ComfyUI 分配的 GPU 设备，绝不写死 'cuda:0' + main_device = model_patcher.load_device + + diffusion_model = getattr(base_model, 'diffusion_model', None) + if not diffusion_model: + return + + unsupported_name = is_unsupported_model(diffusion_model) + if unsupported_name: + print(f"\033[93m[XB-BOX 拦截盾]\033[0m: 模型 ({unsupported_name}) 属于高耦合架构，不支持物理分块。已自动跳过！") + return + + all_blocks = [] + block_paths = [ + 'transformer_blocks', 'blocks', 'down_blocks', 'up_blocks', 'mid_block', + 'layers', 'attention_blocks', 'input_blocks', 'middle_block', 'output_blocks', + 'double_blocks', 'single_blocks', 'joint_blocks' + ] + + for path in block_paths: + if hasattr(diffusion_model, path): + attr = getattr(diffusion_model, path) + if isinstance(attr, (list, torch.nn.ModuleList)): + for item in attr: + all_blocks.append(item) + elif isinstance(attr, torch.nn.Module) and path in ['mid_block', 'middle_block']: + all_blocks.append(attr) + + if not all_blocks: + return + + total_blocks = len(all_blocks) + # 🛡️ 防呆：用户输入不可超过模型总块数 + if blocks_to_swap > total_blocks: + effective = total_blocks + tag = "匹配最大可分块参数" + else: + effective = blocks_to_swap + tag = "匹配用户设置的参数" + print(f"\033[96m[XB UNet Block Swap]\033[0m: 静态物理分块交换已激活！已找到 {total_blocks} 个引擎模块，分割 {effective} 个分块（{tag}）。") + + for b, block in tqdm(enumerate(all_blocks), total=total_blocks, desc="Slicing UNet pipeline"): + if b > effective: + block.to(main_device) + else: + block.to(model_patcher.offload_device) + + mm.soft_empty_cache() + gc.collect() + + unet_model = unet_model.clone() + unet_model.add_callback(CallbacksMP.ON_LOAD, swap_blocks) + + return (unet_model, ) + +# ============================================================ +# XB_CheckpointBlockSwap — Checkpoint 分块交换 (显存优化) +# ============================================================ +class XB_CheckpointBlockSwap: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "checkpoint_model": (anyType, {"tooltip": "Accepts any model input (including CLIP/VAE)."}), + "blocks_to_swap": ("INT", {"default": 15, "min": 0, "max": 1000, "step": 1}), + "offload_img_emb": ("BOOLEAN", {"default": True}), + "offload_txt_emb": ("BOOLEAN", {"default": True}), + "use_non_blocking": ("BOOLEAN", {"default": False}), + }, + } + + RETURN_TYPES = (anyType,) + RETURN_NAMES = ("checkpoint_model",) + CATEGORY = "XB_ToolBox/VRAM_Hacks" + FUNCTION = "set_callback" + + def set_callback(self, checkpoint_model: ModelPatcher, blocks_to_swap, offload_txt_emb, offload_img_emb, use_non_blocking): + if not isinstance(checkpoint_model, ModelPatcher): + return (checkpoint_model,) + + # ── 0 = 关闭分块功能：直接穿透，不做任何检测、不输出动态显存休眠警告 ── + if blocks_to_swap == 0: + print("\033[96m[XB Checkpoint Block Swap]\033[0m: 分块数量 = 0，分块功能已关闭（节点穿透）") + return (checkpoint_model,) + + if is_dynamic_vram_active(): + return (checkpoint_model,) + + def swap_blocks(model_patcher: ModelPatcher, device_to, lowvram_model_memory, force_patch_weights, full_load): + base_model = model_patcher.model + # 🛡️ 动态获取 ComfyUI 分配的 GPU 设备 + main_device = model_patcher.load_device + + diffusion_model = getattr(base_model, 'diffusion_model', None) + if not diffusion_model: + return + + unsupported_name = is_unsupported_model(diffusion_model) + if unsupported_name: + print(f"\033[93m[XB-BOX 拦截盾]\033[0m: 模型 ({unsupported_name}) 属于高耦合架构，不支持物理分块。已自动跳过！") + return + + all_blocks = [] + block_paths = [ + 'transformer_blocks', 'blocks', 'down_blocks', 'up_blocks', 'mid_block', + 'layers', 'attention_blocks', 'input_blocks', 'middle_block', 'output_blocks', + 'double_blocks', 'single_blocks', 'joint_blocks' + ] + + for path in block_paths: + if hasattr(diffusion_model, path): + attr = getattr(diffusion_model, path) + if isinstance(attr, (list, torch.nn.ModuleList)): + for item in attr: + all_blocks.append(item) + elif isinstance(attr, torch.nn.Module) and path in ['mid_block', 'middle_block']: + all_blocks.append(attr) + + if all_blocks: + total_blocks = len(all_blocks) + # 🛡️ 防呆：用户输入不可超过模型总块数 + if blocks_to_swap > total_blocks: + effective = total_blocks + tag = "匹配最大可分块参数" + else: + effective = blocks_to_swap + tag = "匹配用户设置的参数" + print(f"\033[96m[XB Checkpoint Block Swap]\033[0m: 静态物理分块交换已激活！已找到 {total_blocks} 个引擎模块，分割 {effective} 个分块（{tag}）。") + + for b, block in tqdm(enumerate(all_blocks), total=total_blocks, desc="Slicing Checkpoint pipeline"): + if b > effective: + block.to(main_device) + else: + block.to(model_patcher.offload_device) + + if offload_txt_emb: + for path in ['text_embedding', 'caption_encoder', 'text_encoder']: + if hasattr(diffusion_model, path) and getattr(diffusion_model, path) is not None: + getattr(diffusion_model, path).to(model_patcher.offload_device, non_blocking=use_non_blocking) + break + + if offload_img_emb: + for path in ['img_emb', 'image_encoder', 'visual_encoder']: + if hasattr(diffusion_model, path) and getattr(diffusion_model, path) is not None: + getattr(diffusion_model, path).to(model_patcher.offload_device, non_blocking=use_non_blocking) + break + + mm.soft_empty_cache() + gc.collect() + + checkpoint_model = checkpoint_model.clone() + checkpoint_model.add_callback(CallbacksMP.ON_LOAD, swap_blocks) + return (checkpoint_model, ) \ No newline at end of file diff --git a/nodes_blockswap_sage.py b/nodes/core/nodes_blockswap_sage.py similarity index 100% rename from nodes_blockswap_sage.py rename to nodes/core/nodes_blockswap_sage.py diff --git a/nodes_clip_text_edit.py b/nodes/core/nodes_clip_text_edit.py similarity index 100% rename from nodes_clip_text_edit.py rename to nodes/core/nodes_clip_text_edit.py diff --git a/nodes_fast_toggle.py b/nodes/core/nodes_fast_toggle.py similarity index 100% rename from nodes_fast_toggle.py rename to nodes/core/nodes_fast_toggle.py diff --git a/nodes_label.py b/nodes/core/nodes_label.py similarity index 96% rename from nodes_label.py rename to nodes/core/nodes_label.py index fa348be..7136fc0 100644 --- a/nodes_label.py +++ b/nodes/core/nodes_label.py @@ -1,24 +1,24 @@ -""" -XB-BOX Canvas Label — 画布文字标签节点 -====================================== -纯展示型节点，用于在工作流画布上添加说明文字。 -所有渲染逻辑由前端 JS (xb_label.js) 处理。 -""" - - -# ============================================================ -# XB_CanvasLabel — 画布文字标签 -# ============================================================ -class XB_CanvasLabel: - """纯显示标签节点 - 在画布上展示自定义文字，兼容所有 ComfyUI 版本""" - - @classmethod - def INPUT_TYPES(cls): - return {"required": {}} - - RETURN_TYPES = () - FUNCTION = "noop" - CATEGORY = "XB_ToolBox/Display" - - def noop(self, **kwargs): - return () +""" +XB-BOX Canvas Label — 画布文字标签节点 +====================================== +纯展示型节点，用于在工作流画布上添加说明文字。 +所有渲染逻辑由前端 JS (xb_label.js) 处理。 +""" + + +# ============================================================ +# XB_CanvasLabel — 画布文字标签 +# ============================================================ +class XB_CanvasLabel: + """纯显示标签节点 - 在画布上展示自定义文字，兼容所有 ComfyUI 版本""" + + @classmethod + def INPUT_TYPES(cls): + return {"required": {}} + + RETURN_TYPES = () + FUNCTION = "noop" + CATEGORY = "XB_ToolBox/Display" + + def noop(self, **kwargs): + return () diff --git a/nodes_list_dispatcher.py b/nodes/core/nodes_list_dispatcher.py similarity index 100% rename from nodes_list_dispatcher.py rename to nodes/core/nodes_list_dispatcher.py diff --git a/nodes_lora_loader.py b/nodes/core/nodes_lora_loader.py similarity index 100% rename from nodes_lora_loader.py rename to nodes/core/nodes_lora_loader.py diff --git a/nodes_msr.py b/nodes/core/nodes_msr.py similarity index 100% rename from nodes_msr.py rename to nodes/core/nodes_msr.py diff --git a/nodes_reference_any.py b/nodes/core/nodes_reference_any.py similarity index 100% rename from nodes_reference_any.py rename to nodes/core/nodes_reference_any.py diff --git a/nodes_sageatt.py b/nodes/core/nodes_sageatt.py similarity index 97% rename from nodes_sageatt.py rename to nodes/core/nodes_sageatt.py index 5fe1450..62a1c94 100644 --- a/nodes_sageatt.py +++ b/nodes/core/nodes_sageatt.py @@ -1,321 +1,321 @@ -""" -XB-SageAttention Accelerator -============================= -保持原版 KJNodes 设计, 仅添加 GPU 信息日志。 - -工作模式: - - 关闭: 直接透传模型 - - 自动: 委托 KJNodes auto 逻辑 (sageattn 默认参数) - - pytorch attention: 融合官方 ModelAttentionBackend — 原生 SDPA - - comfy kitchen attention: 融合官方 ModelAttentionBackend — INT8 量化注意力 - - Nvidia-Sage: KJNodes auto 逻辑 (N卡) - - AMD-RDNA4-Sage 2.2: KJNodes auto 逻辑 (RDNA4) - - 内置模式 A~D: 自定义 sage_config (M/N/GROUP/WAVE/WARP/NSTAGES) - - 自定模式 A~C: 自定义 sage_config (配合外部调参软件) - -A卡兼容: - - RDNA3: Sage 1.0.6 不支持 sage_config, 选中内置/自定时会 fallback 到 SDPA - - RDNA4: Sage 2.2.0 完全兼容 - - N卡: Sage 2.x 完全兼容 -""" - -import torch -from comfy.ldm.modules.attention import wrap_attn - - -# ── GPU 信息 (仅日志用途) ── -def _is_rocm(): - try: - return torch.cuda.is_available() and hasattr(torch.version, "hip") and torch.version.hip is not None - except Exception: - return False - -def _gpu_label(): - try: - if not torch.cuda.is_available(): - return "CPU" - if _is_rocm(): - arch = "" - try: - raw = torch.cuda.get_device_properties(0).gcnArchName - arch = " " + raw.split(":")[0] if raw else "" - except Exception: - pass - return f"ROCm{arch}" - return "CUDA" - except Exception: - return "Unknown" - -_GPU = _gpu_label() -print(f"[XB-SageAttn] GPU: {_GPU}, Sage节点已就绪") - - -# ══════════════════════════════════════════════════════════ -# XB_SageAttentionAccelerator (保持原版设计) -# ══════════════════════════════════════════════════════════ - -class XB_SageAttentionAccelerator: - """SageAttention 加速补丁节点。 - - 选项说明: - - 关闭: 节点完全不生效，直接透传模型 - - 自动: 一字不差使用 KJNodes 的 auto 逻辑 (sageattn 默认参数) - - pytorch attention: 融合官方 ModelAttentionBackend，使用原生 SDPA - - comfy kitchen attention: 融合官方 ModelAttentionBackend，使用 INT8 量化注意力 - - 内置模式 A~D: 使用指定 sage_config 参数 (M/N/GROUP/WAVE/WARP/NSTAGES) - - 自定模式 A~C: 自定义 sage_config 参数 - """ - - # 融合官方 ModelAttentionBackend 的两个后端 - _OFFICIAL_BACKENDS = ("pytorch attention", "comfy kitchen attention") - - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "model": ("MODEL",), - "preset": ([ - "关闭", - "自动", - "pytorch attention", - "comfy kitchen attention", - "Nvidia-Sage", - "AMD-RDNA4-Sage 2.2", - "内置模式 A (128x128x32)", - "内置模式 B (128x64x96)", - "内置模式 C (128x16x16)", - "内置模式 D (64x64x16)", - "自定模式 A (机智启动器)", - "自定模式 B (机智启动器)", - "自定模式 C (机智启动器)", - ], {"default": "关闭"}), - } - } - - RETURN_TYPES = ("MODEL",) - FUNCTION = "patch" - CATEGORY = "XB_ToolBox/VRAM_Hacks" - - def patch(self, model, preset): - if preset == "关闭": - print(f"\033[96m[XB-SageAttn]\033[0m: 已关闭, 模型透传 (GPU={_GPU})") - return (model,) - - try: - # 融合官方 ModelAttentionBackend - if preset in self._OFFICIAL_BACKENDS: - return (self._apply_official_backend(model, preset),) - return self._apply_patch(model, preset) - except Exception as e: - import traceback - print(f"\n\033[93m[XB-SageAttn 警告] 节点异常，自动跳过！工作流继续运行。\033[0m") - print(f"\033[93m[XB-SageAttn 错误] GPU={_GPU}, preset={preset}\033[0m") - print(f"\033[93m[XB-SageAttn 详情]\033[0m {e}") - traceback.print_exc() - return (model,) - - def _apply_official_backend(self, model, preset): - """融合官方 ModelAttentionBackend 节点 (pytorch / comfy kitchen attention)。""" - import comfy.ldm.modules.attention as attn_mod - - backend_name = "comfy_kitchen_int8" if preset == "comfy kitchen attention" else "pytorch" - attention_function = attn_mod.get_attention_function(backend_name, None) - - if attention_function is None: - print(f"\033[93m[XB-SageAttn]\033[0m: '{preset}' 当前不可用 " - f"(需 Nvidia/AMD GPU 且安装 comfy_kitchen)，已回退 pytorch attention") - attention_function = attn_mod.get_attention_function("pytorch") - preset = "pytorch attention (回退)" - - m = model.clone() - if "transformer_options" not in m.model_options: - m.model_options["transformer_options"] = {} - # 必须用官方 setter: 它会正确传递 container_function (INT8 kernel 依赖) - m.set_model_optimized_attention(attention_function) - - print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m{preset}\033[0m " - f"(官方 ModelAttentionBackend, GPU={_GPU})") - return m - - def _apply_patch(self, model, preset): - # ── 自动：一字不差使用 KJNodes 的 auto 逻辑 ── - if preset in ("自动", "Nvidia-Sage", "AMD-RDNA4-Sage 2.2"): - from sageattention import sageattn - - def sage_func(q, k, v, is_causal=False, attn_mask=None, tensor_layout="NHD"): - return sageattn(q, k, v, is_causal=is_causal, attn_mask=attn_mask, tensor_layout=tensor_layout) - - @wrap_attn - def attention_sage(q, k, v, heads, mask=None, attn_precision=None, - skip_reshape=False, skip_output_reshape=False, **kwargs): - if kwargs.get("low_precision_attention", True) is False: - from comfy.ldm.modules.attention import attention_pytorch - return attention_pytorch(q, k, v, heads, mask=mask, - skip_reshape=skip_reshape, - skip_output_reshape=skip_output_reshape, **kwargs) - in_dtype = v.dtype - if q.dtype == torch.float32 or k.dtype == torch.float32 or v.dtype == torch.float32: - q, k, v = q.to(torch.float16), k.to(torch.float16), v.to(torch.float16) - if skip_reshape: - b, _, _, dim_head = q.shape - tensor_layout = "HND" - else: - b, _, dim_head = q.shape - dim_head //= heads - q, k, v = map( - lambda t: t.view(b, -1, heads, dim_head), - (q, k, v), - ) - tensor_layout = "NHD" - if mask is not None: - if mask.ndim == 2: - mask = mask.unsqueeze(0) - if mask.ndim == 3: - mask = mask.unsqueeze(1) - out = sage_func(q, k, v, attn_mask=mask, is_causal=False, - tensor_layout=tensor_layout).to(in_dtype) - if tensor_layout == "HND": - if not skip_output_reshape: - out = out.transpose(1, 2).reshape(b, -1, heads * dim_head) - else: - if skip_output_reshape: - out = out.transpose(1, 2) - else: - out = out.reshape(b, -1, heads * dim_head) - return out - - m = model.clone() - if "transformer_options" not in m.model_options: - m.model_options["transformer_options"] = {} - m.model_options["transformer_options"]["optimized_attention_override"] = \ - lambda func, *args, **kwargs: attention_sage.__wrapped__(*args, **kwargs) - - print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m自动 (KJNodes auto)\033[0m (GPU={_GPU})") - return (m,) - - # ── 内置/自定模式：使用指定 sage_config 参数 ── - configs = { - "内置模式 A (128x128x32)": {'M': 128, 'N': 128, 'GROUP': 32, 'WAVE': 2, 'WARP': 8, 'NSTAGES': 1}, - "内置模式 B (128x64x96)": {'M': 128, 'N': 64, 'GROUP': 96, 'WAVE': 3, 'WARP': 8, 'NSTAGES': 2}, - "内置模式 C (128x16x16)": {'M': 128, 'N': 16, 'GROUP': 16, 'WAVE': 2, 'WARP': 4, 'NSTAGES': 2}, - "内置模式 D (64x64x16)": {'M': 64, 'N': 64, 'GROUP': 16, 'WAVE': 4, 'WARP': 4, 'NSTAGES': 2}, - "自定模式 A (机智启动器)": {'M': 128, 'N': 128, 'GROUP': 16, 'WAVE': 4, 'WARP': 8, 'NSTAGES': 1}, - "自定模式 B (机智启动器)": {'M': 128, 'N': 16, 'GROUP': 8, 'WAVE': 2, 'WARP': 8, 'NSTAGES': 2}, - "自定模式 C (机智启动器)": {'M': 64, 'N': 128, 'GROUP': 96, 'WAVE': 2, 'WARP': 2, 'NSTAGES': 1}, - } - - selected_cfg = configs[preset] - from sageattention import sageattn - - _warned = {} - - def attention_override_sage(func, q, k, v, heads, mask=None, attn_precision=None, - skip_reshape=False, skip_output_reshape=False, **kwargs): - in_dtype = v.dtype - - # 3D/4D 维度校验 - if q.ndim != 3 and not skip_reshape: - return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, - skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) - - if skip_reshape: - b, _, _, dim_head = q.shape - tensor_layout = "HND" - else: - b, _, d = q.shape - dim_head = d // heads - tensor_layout = "NHD" - - if mask is not None: - if "mask" not in _warned: - print("\033[93m[SageAttn]\033[0m: Attention mask detected — falling back to SDPA.") - _warned["mask"] = True - return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, - skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) - - # 动态补零策略 - _STANDARD_DIMS = (16, 32, 64, 128, 256) - _orig_dim_head = dim_head - _pad_target = None - if dim_head not in _STANDARD_DIMS: - for _sd in _STANDARD_DIMS: - if _sd >= dim_head: - _pad_target = _sd - break - if _pad_target is not None: - if "pad" not in _warned: - print("\033[96m[SageAttn]\033[0m: Padding dim_head {} → {} for SageAttention compatibility." - .format(dim_head, _pad_target)) - _warned["pad"] = True - _scale = (float(_pad_target) / float(_orig_dim_head)) ** 0.25 - else: - if "large_dim" not in _warned: - print("\033[93m[SageAttn]\033[0m: dim_head ({}) exceeds max supported — falling back to SDPA." - .format(_orig_dim_head)) - _warned["large_dim"] = True - return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, - skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) - - if q.dtype == torch.float32 or k.dtype == torch.float32 or v.dtype == torch.float32: - q, k, v = q.to(torch.float16), k.to(torch.float16), v.to(torch.float16) - - if not skip_reshape: - q = q.view(b, -1, heads, _orig_dim_head) - k = k.view(b, -1, heads, _orig_dim_head) - v = v.view(b, -1, heads, _orig_dim_head) - - _orig_q, _orig_k, _orig_v = q, k, v - - if _pad_target is not None: - _pad_len = _pad_target - _orig_dim_head - q = torch.nn.functional.pad(q, (0, _pad_len)) - k = torch.nn.functional.pad(k, (0, _pad_len)) - v = torch.nn.functional.pad(v, (0, _pad_len)) - q = q * _scale - k = k * _scale - dim_head = _pad_target - - is_causal = kwargs.get("is_causal", False) - try: - out = sageattn(q, k, v, is_causal=is_causal, tensor_layout=tensor_layout, - sage_config=selected_cfg).to(in_dtype) - except Exception as e: - if "sage_fallback" not in _warned: - print("\033[93m[SageAttn]\033[0m: SageAttention kernel failed (dim_head={}) — falling back to SDPA. " - "Error: {}".format(_orig_dim_head, str(e)[:120])) - _warned["sage_fallback"] = True - if skip_reshape: - return func(_orig_q, _orig_k, _orig_v, heads, mask=mask, attn_precision=attn_precision, - skip_reshape=True, skip_output_reshape=skip_output_reshape, **kwargs) - else: - q_3d = _orig_q.reshape(b, -1, heads * _orig_dim_head) - k_3d = _orig_k.reshape(b, -1, heads * _orig_dim_head) - v_3d = _orig_v.reshape(b, -1, heads * _orig_dim_head) - return func(q_3d, k_3d, v_3d, heads, mask=mask, attn_precision=attn_precision, - skip_reshape=False, skip_output_reshape=False, **kwargs) - - if _pad_target is not None: - out = out[..., :_orig_dim_head] - dim_head = _orig_dim_head - - if tensor_layout == "HND": - if not skip_output_reshape: - out = out.transpose(1, 2).reshape(b, -1, heads * dim_head) - else: - if skip_output_reshape: - out = out.transpose(1, 2) - else: - out = out.reshape(b, -1, heads * dim_head) - - return out - - m = model.clone() - - if "transformer_options" not in m.model_options: - m.model_options["transformer_options"] = {} - - m.model_options["transformer_options"]["optimized_attention_override"] = attention_override_sage - - print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m{preset}\033[0m (GPU={_GPU})") - - return (m,) +""" +XB-SageAttention Accelerator +============================= +保持原版 KJNodes 设计, 仅添加 GPU 信息日志。 + +工作模式: + - 关闭: 直接透传模型 + - 自动: 委托 KJNodes auto 逻辑 (sageattn 默认参数) + - pytorch attention: 融合官方 ModelAttentionBackend — 原生 SDPA + - comfy kitchen attention: 融合官方 ModelAttentionBackend — INT8 量化注意力 + - Nvidia-Sage: KJNodes auto 逻辑 (N卡) + - AMD-RDNA4-Sage 2.2: KJNodes auto 逻辑 (RDNA4) + - 内置模式 A~D: 自定义 sage_config (M/N/GROUP/WAVE/WARP/NSTAGES) + - 自定模式 A~C: 自定义 sage_config (配合外部调参软件) + +A卡兼容: + - RDNA3: Sage 1.0.6 不支持 sage_config, 选中内置/自定时会 fallback 到 SDPA + - RDNA4: Sage 2.2.0 完全兼容 + - N卡: Sage 2.x 完全兼容 +""" + +import torch +from comfy.ldm.modules.attention import wrap_attn + + +# ── GPU 信息 (仅日志用途) ── +def _is_rocm(): + try: + return torch.cuda.is_available() and hasattr(torch.version, "hip") and torch.version.hip is not None + except Exception: + return False + +def _gpu_label(): + try: + if not torch.cuda.is_available(): + return "CPU" + if _is_rocm(): + arch = "" + try: + raw = torch.cuda.get_device_properties(0).gcnArchName + arch = " " + raw.split(":")[0] if raw else "" + except Exception: + pass + return f"ROCm{arch}" + return "CUDA" + except Exception: + return "Unknown" + +_GPU = _gpu_label() +print(f"[XB-SageAttn] GPU: {_GPU}, Sage节点已就绪") + + +# ══════════════════════════════════════════════════════════ +# XB_SageAttentionAccelerator (保持原版设计) +# ══════════════════════════════════════════════════════════ + +class XB_SageAttentionAccelerator: + """SageAttention 加速补丁节点。 + + 选项说明: + - 关闭: 节点完全不生效，直接透传模型 + - 自动: 一字不差使用 KJNodes 的 auto 逻辑 (sageattn 默认参数) + - pytorch attention: 融合官方 ModelAttentionBackend，使用原生 SDPA + - comfy kitchen attention: 融合官方 ModelAttentionBackend，使用 INT8 量化注意力 + - 内置模式 A~D: 使用指定 sage_config 参数 (M/N/GROUP/WAVE/WARP/NSTAGES) + - 自定模式 A~C: 自定义 sage_config 参数 + """ + + # 融合官方 ModelAttentionBackend 的两个后端 + _OFFICIAL_BACKENDS = ("pytorch attention", "comfy kitchen attention") + + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "model": ("MODEL",), + "preset": ([ + "关闭", + "自动", + "pytorch attention", + "comfy kitchen attention", + "Nvidia-Sage", + "AMD-RDNA4-Sage 2.2", + "内置模式 A (128x128x32)", + "内置模式 B (128x64x96)", + "内置模式 C (128x16x16)", + "内置模式 D (64x64x16)", + "自定模式 A (机智启动器)", + "自定模式 B (机智启动器)", + "自定模式 C (机智启动器)", + ], {"default": "关闭"}), + } + } + + RETURN_TYPES = ("MODEL",) + FUNCTION = "patch" + CATEGORY = "XB_ToolBox/VRAM_Hacks" + + def patch(self, model, preset): + if preset == "关闭": + print(f"\033[96m[XB-SageAttn]\033[0m: 已关闭, 模型透传 (GPU={_GPU})") + return (model,) + + try: + # 融合官方 ModelAttentionBackend + if preset in self._OFFICIAL_BACKENDS: + return (self._apply_official_backend(model, preset),) + return self._apply_patch(model, preset) + except Exception as e: + import traceback + print(f"\n\033[93m[XB-SageAttn 警告] 节点异常，自动跳过！工作流继续运行。\033[0m") + print(f"\033[93m[XB-SageAttn 错误] GPU={_GPU}, preset={preset}\033[0m") + print(f"\033[93m[XB-SageAttn 详情]\033[0m {e}") + traceback.print_exc() + return (model,) + + def _apply_official_backend(self, model, preset): + """融合官方 ModelAttentionBackend 节点 (pytorch / comfy kitchen attention)。""" + import comfy.ldm.modules.attention as attn_mod + + backend_name = "comfy_kitchen_int8" if preset == "comfy kitchen attention" else "pytorch" + attention_function = attn_mod.get_attention_function(backend_name, None) + + if attention_function is None: + print(f"\033[93m[XB-SageAttn]\033[0m: '{preset}' 当前不可用 " + f"(需 Nvidia/AMD GPU 且安装 comfy_kitchen)，已回退 pytorch attention") + attention_function = attn_mod.get_attention_function("pytorch") + preset = "pytorch attention (回退)" + + m = model.clone() + if "transformer_options" not in m.model_options: + m.model_options["transformer_options"] = {} + # 必须用官方 setter: 它会正确传递 container_function (INT8 kernel 依赖) + m.set_model_optimized_attention(attention_function) + + print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m{preset}\033[0m " + f"(官方 ModelAttentionBackend, GPU={_GPU})") + return m + + def _apply_patch(self, model, preset): + # ── 自动：一字不差使用 KJNodes 的 auto 逻辑 ── + if preset in ("自动", "Nvidia-Sage", "AMD-RDNA4-Sage 2.2"): + from sageattention import sageattn + + def sage_func(q, k, v, is_causal=False, attn_mask=None, tensor_layout="NHD"): + return sageattn(q, k, v, is_causal=is_causal, attn_mask=attn_mask, tensor_layout=tensor_layout) + + @wrap_attn + def attention_sage(q, k, v, heads, mask=None, attn_precision=None, + skip_reshape=False, skip_output_reshape=False, **kwargs): + if kwargs.get("low_precision_attention", True) is False: + from comfy.ldm.modules.attention import attention_pytorch + return attention_pytorch(q, k, v, heads, mask=mask, + skip_reshape=skip_reshape, + skip_output_reshape=skip_output_reshape, **kwargs) + in_dtype = v.dtype + if q.dtype == torch.float32 or k.dtype == torch.float32 or v.dtype == torch.float32: + q, k, v = q.to(torch.float16), k.to(torch.float16), v.to(torch.float16) + if skip_reshape: + b, _, _, dim_head = q.shape + tensor_layout = "HND" + else: + b, _, dim_head = q.shape + dim_head //= heads + q, k, v = map( + lambda t: t.view(b, -1, heads, dim_head), + (q, k, v), + ) + tensor_layout = "NHD" + if mask is not None: + if mask.ndim == 2: + mask = mask.unsqueeze(0) + if mask.ndim == 3: + mask = mask.unsqueeze(1) + out = sage_func(q, k, v, attn_mask=mask, is_causal=False, + tensor_layout=tensor_layout).to(in_dtype) + if tensor_layout == "HND": + if not skip_output_reshape: + out = out.transpose(1, 2).reshape(b, -1, heads * dim_head) + else: + if skip_output_reshape: + out = out.transpose(1, 2) + else: + out = out.reshape(b, -1, heads * dim_head) + return out + + m = model.clone() + if "transformer_options" not in m.model_options: + m.model_options["transformer_options"] = {} + m.model_options["transformer_options"]["optimized_attention_override"] = \ + lambda func, *args, **kwargs: attention_sage.__wrapped__(*args, **kwargs) + + print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m自动 (KJNodes auto)\033[0m (GPU={_GPU})") + return (m,) + + # ── 内置/自定模式：使用指定 sage_config 参数 ── + configs = { + "内置模式 A (128x128x32)": {'M': 128, 'N': 128, 'GROUP': 32, 'WAVE': 2, 'WARP': 8, 'NSTAGES': 1}, + "内置模式 B (128x64x96)": {'M': 128, 'N': 64, 'GROUP': 96, 'WAVE': 3, 'WARP': 8, 'NSTAGES': 2}, + "内置模式 C (128x16x16)": {'M': 128, 'N': 16, 'GROUP': 16, 'WAVE': 2, 'WARP': 4, 'NSTAGES': 2}, + "内置模式 D (64x64x16)": {'M': 64, 'N': 64, 'GROUP': 16, 'WAVE': 4, 'WARP': 4, 'NSTAGES': 2}, + "自定模式 A (机智启动器)": {'M': 128, 'N': 128, 'GROUP': 16, 'WAVE': 4, 'WARP': 8, 'NSTAGES': 1}, + "自定模式 B (机智启动器)": {'M': 128, 'N': 16, 'GROUP': 8, 'WAVE': 2, 'WARP': 8, 'NSTAGES': 2}, + "自定模式 C (机智启动器)": {'M': 64, 'N': 128, 'GROUP': 96, 'WAVE': 2, 'WARP': 2, 'NSTAGES': 1}, + } + + selected_cfg = configs[preset] + from sageattention import sageattn + + _warned = {} + + def attention_override_sage(func, q, k, v, heads, mask=None, attn_precision=None, + skip_reshape=False, skip_output_reshape=False, **kwargs): + in_dtype = v.dtype + + # 3D/4D 维度校验 + if q.ndim != 3 and not skip_reshape: + return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, + skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) + + if skip_reshape: + b, _, _, dim_head = q.shape + tensor_layout = "HND" + else: + b, _, d = q.shape + dim_head = d // heads + tensor_layout = "NHD" + + if mask is not None: + if "mask" not in _warned: + print("\033[93m[SageAttn]\033[0m: Attention mask detected — falling back to SDPA.") + _warned["mask"] = True + return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, + skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) + + # 动态补零策略 + _STANDARD_DIMS = (16, 32, 64, 128, 256) + _orig_dim_head = dim_head + _pad_target = None + if dim_head not in _STANDARD_DIMS: + for _sd in _STANDARD_DIMS: + if _sd >= dim_head: + _pad_target = _sd + break + if _pad_target is not None: + if "pad" not in _warned: + print("\033[96m[SageAttn]\033[0m: Padding dim_head {} → {} for SageAttention compatibility." + .format(dim_head, _pad_target)) + _warned["pad"] = True + _scale = (float(_pad_target) / float(_orig_dim_head)) ** 0.25 + else: + if "large_dim" not in _warned: + print("\033[93m[SageAttn]\033[0m: dim_head ({}) exceeds max supported — falling back to SDPA." + .format(_orig_dim_head)) + _warned["large_dim"] = True + return func(q, k, v, heads, mask=mask, attn_precision=attn_precision, + skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs) + + if q.dtype == torch.float32 or k.dtype == torch.float32 or v.dtype == torch.float32: + q, k, v = q.to(torch.float16), k.to(torch.float16), v.to(torch.float16) + + if not skip_reshape: + q = q.view(b, -1, heads, _orig_dim_head) + k = k.view(b, -1, heads, _orig_dim_head) + v = v.view(b, -1, heads, _orig_dim_head) + + _orig_q, _orig_k, _orig_v = q, k, v + + if _pad_target is not None: + _pad_len = _pad_target - _orig_dim_head + q = torch.nn.functional.pad(q, (0, _pad_len)) + k = torch.nn.functional.pad(k, (0, _pad_len)) + v = torch.nn.functional.pad(v, (0, _pad_len)) + q = q * _scale + k = k * _scale + dim_head = _pad_target + + is_causal = kwargs.get("is_causal", False) + try: + out = sageattn(q, k, v, is_causal=is_causal, tensor_layout=tensor_layout, + sage_config=selected_cfg).to(in_dtype) + except Exception as e: + if "sage_fallback" not in _warned: + print("\033[93m[SageAttn]\033[0m: SageAttention kernel failed (dim_head={}) — falling back to SDPA. " + "Error: {}".format(_orig_dim_head, str(e)[:120])) + _warned["sage_fallback"] = True + if skip_reshape: + return func(_orig_q, _orig_k, _orig_v, heads, mask=mask, attn_precision=attn_precision, + skip_reshape=True, skip_output_reshape=skip_output_reshape, **kwargs) + else: + q_3d = _orig_q.reshape(b, -1, heads * _orig_dim_head) + k_3d = _orig_k.reshape(b, -1, heads * _orig_dim_head) + v_3d = _orig_v.reshape(b, -1, heads * _orig_dim_head) + return func(q_3d, k_3d, v_3d, heads, mask=mask, attn_precision=attn_precision, + skip_reshape=False, skip_output_reshape=False, **kwargs) + + if _pad_target is not None: + out = out[..., :_orig_dim_head] + dim_head = _orig_dim_head + + if tensor_layout == "HND": + if not skip_output_reshape: + out = out.transpose(1, 2).reshape(b, -1, heads * dim_head) + else: + if skip_output_reshape: + out = out.transpose(1, 2) + else: + out = out.reshape(b, -1, heads * dim_head) + + return out + + m = model.clone() + + if "transformer_options" not in m.model_options: + m.model_options["transformer_options"] = {} + + m.model_options["transformer_options"]["optimized_attention_override"] = attention_override_sage + + print(f"\033[96m[XB-SageAttn]\033[0m: 引擎 → \033[92m{preset}\033[0m (GPU={_GPU})") + + return (m,) diff --git a/nodes_status_switch.py b/nodes/core/nodes_status_switch.py similarity index 100% rename from nodes_status_switch.py rename to nodes/core/nodes_status_switch.py diff --git a/nodes_string_merge.py b/nodes/core/nodes_string_merge.py similarity index 100% rename from nodes_string_merge.py rename to nodes/core/nodes_string_merge.py diff --git a/nodes_super.py b/nodes/core/nodes_super.py similarity index 100% rename from nodes_super.py rename to nodes/core/nodes_super.py diff --git a/nodes_vanilla_wrappers.py b/nodes/core/nodes_vanilla_wrappers.py similarity index 100% rename from nodes_vanilla_wrappers.py rename to nodes/core/nodes_vanilla_wrappers.py diff --git a/nodes_wiring.py b/nodes/core/nodes_wiring.py similarity index 96% rename from nodes_wiring.py rename to nodes/core/nodes_wiring.py index f91c708..bfe91c1 100644 --- a/nodes_wiring.py +++ b/nodes/core/nodes_wiring.py @@ -1,46 +1,46 @@ -import folder_paths - -class AnyType(str): - def __ne__(self, __value: object) -> bool: - return False -anyType = AnyType("*") - -# ============================================================ -# XB_UNetNameBroadcaster — UNet 名称广播器 -# ============================================================ -class XB_UNetNameBroadcaster: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "unet_name": (folder_paths.get_filename_list("unet"), ), - } - } - - RETURN_TYPES = (anyType,) - RETURN_NAMES = ("UNet_Name",) - FUNCTION = "broadcast" - CATEGORY = "XB_ToolBox/Wiring" - - def broadcast(self, unet_name): - return (unet_name,) - -# ============================================================ -# XB_CLIPNameBroadcaster — CLIP 名称广播器 -# ============================================================ -class XB_CLIPNameBroadcaster: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "clip_name": (folder_paths.get_filename_list("clip"), ), - } - } - - RETURN_TYPES = (anyType,) - RETURN_NAMES = ("CLIP_Name",) - FUNCTION = "broadcast" - CATEGORY = "XB_ToolBox/Wiring" - - def broadcast(self, clip_name): +import folder_paths + +class AnyType(str): + def __ne__(self, __value: object) -> bool: + return False +anyType = AnyType("*") + +# ============================================================ +# XB_UNetNameBroadcaster — UNet 名称广播器 +# ============================================================ +class XB_UNetNameBroadcaster: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "unet_name": (folder_paths.get_filename_list("unet"), ), + } + } + + RETURN_TYPES = (anyType,) + RETURN_NAMES = ("UNet_Name",) + FUNCTION = "broadcast" + CATEGORY = "XB_ToolBox/Wiring" + + def broadcast(self, unet_name): + return (unet_name,) + +# ============================================================ +# XB_CLIPNameBroadcaster — CLIP 名称广播器 +# ============================================================ +class XB_CLIPNameBroadcaster: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "clip_name": (folder_paths.get_filename_list("clip"), ), + } + } + + RETURN_TYPES = (anyType,) + RETURN_NAMES = ("CLIP_Name",) + FUNCTION = "broadcast" + CATEGORY = "XB_ToolBox/Wiring" + + def broadcast(self, clip_name): return (clip_name,) \ No newline at end of file diff --git a/cosyvoice3/__init__.py b/nodes/cosyvoice3/__init__.py similarity index 100% rename from cosyvoice3/__init__.py rename to nodes/cosyvoice3/__init__.py diff --git a/cosyvoice3/assets/workflow_preview.png b/nodes/cosyvoice3/assets/workflow_preview.png similarity index 100% rename from cosyvoice3/assets/workflow_preview.png rename to nodes/cosyvoice3/assets/workflow_preview.png diff --git a/cosyvoice3/cosyvoice/__init__.py b/nodes/cosyvoice3/cosyvoice/__init__.py similarity index 100% rename from cosyvoice3/cosyvoice/__init__.py rename to nodes/cosyvoice3/cosyvoice/__init__.py diff --git a/cosyvoice3/cosyvoice/bin/average_model.py b/nodes/cosyvoice3/cosyvoice/bin/average_model.py similarity index 100% rename from cosyvoice3/cosyvoice/bin/average_model.py rename to nodes/cosyvoice3/cosyvoice/bin/average_model.py diff --git a/cosyvoice3/cosyvoice/bin/export_jit.py b/nodes/cosyvoice3/cosyvoice/bin/export_jit.py similarity index 100% rename from cosyvoice3/cosyvoice/bin/export_jit.py rename to nodes/cosyvoice3/cosyvoice/bin/export_jit.py diff --git a/cosyvoice3/cosyvoice/bin/export_onnx.py b/nodes/cosyvoice3/cosyvoice/bin/export_onnx.py similarity index 100% rename from cosyvoice3/cosyvoice/bin/export_onnx.py rename to nodes/cosyvoice3/cosyvoice/bin/export_onnx.py diff --git a/cosyvoice3/cosyvoice/bin/train.py b/nodes/cosyvoice3/cosyvoice/bin/train.py similarity index 100% rename from cosyvoice3/cosyvoice/bin/train.py rename to nodes/cosyvoice3/cosyvoice/bin/train.py diff --git a/nodes/cosyvoice3/cosyvoice/cli/__init__.py b/nodes/cosyvoice3/cosyvoice/cli/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/cosyvoice3/cosyvoice/cli/cosyvoice.py b/nodes/cosyvoice3/cosyvoice/cli/cosyvoice.py similarity index 100% rename from cosyvoice3/cosyvoice/cli/cosyvoice.py rename to nodes/cosyvoice3/cosyvoice/cli/cosyvoice.py diff --git a/cosyvoice3/cosyvoice/cli/frontend.py b/nodes/cosyvoice3/cosyvoice/cli/frontend.py similarity index 100% rename from cosyvoice3/cosyvoice/cli/frontend.py rename to nodes/cosyvoice3/cosyvoice/cli/frontend.py diff --git a/cosyvoice3/cosyvoice/cli/model.py b/nodes/cosyvoice3/cosyvoice/cli/model.py similarity index 100% rename from cosyvoice3/cosyvoice/cli/model.py rename to nodes/cosyvoice3/cosyvoice/cli/model.py diff --git a/nodes/cosyvoice3/cosyvoice/dataset/__init__.py b/nodes/cosyvoice3/cosyvoice/dataset/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/cosyvoice3/cosyvoice/dataset/dataset.py b/nodes/cosyvoice3/cosyvoice/dataset/dataset.py similarity index 100% rename from cosyvoice3/cosyvoice/dataset/dataset.py rename to nodes/cosyvoice3/cosyvoice/dataset/dataset.py diff --git a/cosyvoice3/cosyvoice/dataset/processor.py b/nodes/cosyvoice3/cosyvoice/dataset/processor.py similarity index 100% rename from cosyvoice3/cosyvoice/dataset/processor.py rename to nodes/cosyvoice3/cosyvoice/dataset/processor.py diff --git a/cosyvoice3/cosyvoice/flow/DiT/dit.py b/nodes/cosyvoice3/cosyvoice/flow/DiT/dit.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/DiT/dit.py rename to nodes/cosyvoice3/cosyvoice/flow/DiT/dit.py diff --git a/cosyvoice3/cosyvoice/flow/DiT/modules.py b/nodes/cosyvoice3/cosyvoice/flow/DiT/modules.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/DiT/modules.py rename to nodes/cosyvoice3/cosyvoice/flow/DiT/modules.py diff --git a/cosyvoice3/cosyvoice/flow/decoder.py b/nodes/cosyvoice3/cosyvoice/flow/decoder.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/decoder.py rename to nodes/cosyvoice3/cosyvoice/flow/decoder.py diff --git a/cosyvoice3/cosyvoice/flow/flow.py b/nodes/cosyvoice3/cosyvoice/flow/flow.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/flow.py rename to nodes/cosyvoice3/cosyvoice/flow/flow.py diff --git a/cosyvoice3/cosyvoice/flow/flow_matching.py b/nodes/cosyvoice3/cosyvoice/flow/flow_matching.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/flow_matching.py rename to nodes/cosyvoice3/cosyvoice/flow/flow_matching.py diff --git a/cosyvoice3/cosyvoice/flow/length_regulator.py b/nodes/cosyvoice3/cosyvoice/flow/length_regulator.py similarity index 100% rename from cosyvoice3/cosyvoice/flow/length_regulator.py rename to nodes/cosyvoice3/cosyvoice/flow/length_regulator.py diff --git a/cosyvoice3/cosyvoice/hifigan/discriminator.py b/nodes/cosyvoice3/cosyvoice/hifigan/discriminator.py similarity index 100% rename from cosyvoice3/cosyvoice/hifigan/discriminator.py rename to nodes/cosyvoice3/cosyvoice/hifigan/discriminator.py diff --git a/cosyvoice3/cosyvoice/hifigan/f0_predictor.py b/nodes/cosyvoice3/cosyvoice/hifigan/f0_predictor.py similarity index 100% rename from cosyvoice3/cosyvoice/hifigan/f0_predictor.py rename to nodes/cosyvoice3/cosyvoice/hifigan/f0_predictor.py diff --git a/cosyvoice3/cosyvoice/hifigan/generator.py b/nodes/cosyvoice3/cosyvoice/hifigan/generator.py similarity index 100% rename from cosyvoice3/cosyvoice/hifigan/generator.py rename to nodes/cosyvoice3/cosyvoice/hifigan/generator.py diff --git a/cosyvoice3/cosyvoice/hifigan/hifigan.py b/nodes/cosyvoice3/cosyvoice/hifigan/hifigan.py similarity index 100% rename from cosyvoice3/cosyvoice/hifigan/hifigan.py rename to nodes/cosyvoice3/cosyvoice/hifigan/hifigan.py diff --git a/cosyvoice3/cosyvoice/llm/llm.py b/nodes/cosyvoice3/cosyvoice/llm/llm.py similarity index 100% rename from cosyvoice3/cosyvoice/llm/llm.py rename to nodes/cosyvoice3/cosyvoice/llm/llm.py diff --git a/cosyvoice3/cosyvoice/tokenizer/assets/multilingual_zh_ja_yue_char_del.tiktoken b/nodes/cosyvoice3/cosyvoice/tokenizer/assets/multilingual_zh_ja_yue_char_del.tiktoken similarity index 100% rename from cosyvoice3/cosyvoice/tokenizer/assets/multilingual_zh_ja_yue_char_del.tiktoken rename to nodes/cosyvoice3/cosyvoice/tokenizer/assets/multilingual_zh_ja_yue_char_del.tiktoken diff --git a/cosyvoice3/cosyvoice/tokenizer/tokenizer.py b/nodes/cosyvoice3/cosyvoice/tokenizer/tokenizer.py similarity index 100% rename from cosyvoice3/cosyvoice/tokenizer/tokenizer.py rename to nodes/cosyvoice3/cosyvoice/tokenizer/tokenizer.py diff --git a/nodes/cosyvoice3/cosyvoice/transformer/__init__.py b/nodes/cosyvoice3/cosyvoice/transformer/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/cosyvoice3/cosyvoice/transformer/activation.py b/nodes/cosyvoice3/cosyvoice/transformer/activation.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/activation.py rename to nodes/cosyvoice3/cosyvoice/transformer/activation.py diff --git a/cosyvoice3/cosyvoice/transformer/attention.py b/nodes/cosyvoice3/cosyvoice/transformer/attention.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/attention.py rename to nodes/cosyvoice3/cosyvoice/transformer/attention.py diff --git a/cosyvoice3/cosyvoice/transformer/convolution.py b/nodes/cosyvoice3/cosyvoice/transformer/convolution.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/convolution.py rename to nodes/cosyvoice3/cosyvoice/transformer/convolution.py diff --git a/cosyvoice3/cosyvoice/transformer/decoder.py b/nodes/cosyvoice3/cosyvoice/transformer/decoder.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/decoder.py rename to nodes/cosyvoice3/cosyvoice/transformer/decoder.py diff --git a/cosyvoice3/cosyvoice/transformer/decoder_layer.py b/nodes/cosyvoice3/cosyvoice/transformer/decoder_layer.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/decoder_layer.py rename to nodes/cosyvoice3/cosyvoice/transformer/decoder_layer.py diff --git a/cosyvoice3/cosyvoice/transformer/embedding.py b/nodes/cosyvoice3/cosyvoice/transformer/embedding.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/embedding.py rename to nodes/cosyvoice3/cosyvoice/transformer/embedding.py diff --git a/cosyvoice3/cosyvoice/transformer/encoder.py b/nodes/cosyvoice3/cosyvoice/transformer/encoder.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/encoder.py rename to nodes/cosyvoice3/cosyvoice/transformer/encoder.py diff --git a/cosyvoice3/cosyvoice/transformer/encoder_layer.py b/nodes/cosyvoice3/cosyvoice/transformer/encoder_layer.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/encoder_layer.py rename to nodes/cosyvoice3/cosyvoice/transformer/encoder_layer.py diff --git a/cosyvoice3/cosyvoice/transformer/label_smoothing_loss.py b/nodes/cosyvoice3/cosyvoice/transformer/label_smoothing_loss.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/label_smoothing_loss.py rename to nodes/cosyvoice3/cosyvoice/transformer/label_smoothing_loss.py diff --git a/cosyvoice3/cosyvoice/transformer/positionwise_feed_forward.py b/nodes/cosyvoice3/cosyvoice/transformer/positionwise_feed_forward.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/positionwise_feed_forward.py rename to nodes/cosyvoice3/cosyvoice/transformer/positionwise_feed_forward.py diff --git a/cosyvoice3/cosyvoice/transformer/subsampling.py b/nodes/cosyvoice3/cosyvoice/transformer/subsampling.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/subsampling.py rename to nodes/cosyvoice3/cosyvoice/transformer/subsampling.py diff --git a/cosyvoice3/cosyvoice/transformer/upsample_encoder.py b/nodes/cosyvoice3/cosyvoice/transformer/upsample_encoder.py similarity index 100% rename from cosyvoice3/cosyvoice/transformer/upsample_encoder.py rename to nodes/cosyvoice3/cosyvoice/transformer/upsample_encoder.py diff --git a/nodes/cosyvoice3/cosyvoice/utils/__init__.py b/nodes/cosyvoice3/cosyvoice/utils/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/cosyvoice3/cosyvoice/utils/class_utils.py b/nodes/cosyvoice3/cosyvoice/utils/class_utils.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/class_utils.py rename to nodes/cosyvoice3/cosyvoice/utils/class_utils.py diff --git a/cosyvoice3/cosyvoice/utils/common.py b/nodes/cosyvoice3/cosyvoice/utils/common.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/common.py rename to nodes/cosyvoice3/cosyvoice/utils/common.py diff --git a/cosyvoice3/cosyvoice/utils/executor.py b/nodes/cosyvoice3/cosyvoice/utils/executor.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/executor.py rename to nodes/cosyvoice3/cosyvoice/utils/executor.py diff --git a/cosyvoice3/cosyvoice/utils/file_utils.py b/nodes/cosyvoice3/cosyvoice/utils/file_utils.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/file_utils.py rename to nodes/cosyvoice3/cosyvoice/utils/file_utils.py diff --git a/cosyvoice3/cosyvoice/utils/frontend_utils.py b/nodes/cosyvoice3/cosyvoice/utils/frontend_utils.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/frontend_utils.py rename to nodes/cosyvoice3/cosyvoice/utils/frontend_utils.py diff --git a/cosyvoice3/cosyvoice/utils/losses.py b/nodes/cosyvoice3/cosyvoice/utils/losses.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/losses.py rename to nodes/cosyvoice3/cosyvoice/utils/losses.py diff --git a/cosyvoice3/cosyvoice/utils/mask.py b/nodes/cosyvoice3/cosyvoice/utils/mask.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/mask.py rename to nodes/cosyvoice3/cosyvoice/utils/mask.py diff --git a/cosyvoice3/cosyvoice/utils/scheduler.py b/nodes/cosyvoice3/cosyvoice/utils/scheduler.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/scheduler.py rename to nodes/cosyvoice3/cosyvoice/utils/scheduler.py diff --git a/cosyvoice3/cosyvoice/utils/train_utils.py b/nodes/cosyvoice3/cosyvoice/utils/train_utils.py similarity index 100% rename from cosyvoice3/cosyvoice/utils/train_utils.py rename to nodes/cosyvoice3/cosyvoice/utils/train_utils.py diff --git a/cosyvoice3/cosyvoice/vllm/cosyvoice2.py b/nodes/cosyvoice3/cosyvoice/vllm/cosyvoice2.py similarity index 100% rename from cosyvoice3/cosyvoice/vllm/cosyvoice2.py rename to nodes/cosyvoice3/cosyvoice/vllm/cosyvoice2.py diff --git a/cosyvoice3/matcha/__init__.py b/nodes/cosyvoice3/matcha/__init__.py similarity index 100% rename from cosyvoice3/matcha/__init__.py rename to nodes/cosyvoice3/matcha/__init__.py diff --git a/cosyvoice3/matcha/hifigan/__init__.py b/nodes/cosyvoice3/matcha/hifigan/__init__.py similarity index 100% rename from cosyvoice3/matcha/hifigan/__init__.py rename to nodes/cosyvoice3/matcha/hifigan/__init__.py diff --git a/cosyvoice3/matcha/hifigan/models.py b/nodes/cosyvoice3/matcha/hifigan/models.py similarity index 100% rename from cosyvoice3/matcha/hifigan/models.py rename to nodes/cosyvoice3/matcha/hifigan/models.py diff --git a/cosyvoice3/matcha/hifigan/xutils.py b/nodes/cosyvoice3/matcha/hifigan/xutils.py similarity index 100% rename from cosyvoice3/matcha/hifigan/xutils.py rename to nodes/cosyvoice3/matcha/hifigan/xutils.py diff --git a/cosyvoice3/matcha/models/__init__.py b/nodes/cosyvoice3/matcha/models/__init__.py similarity index 100% rename from cosyvoice3/matcha/models/__init__.py rename to nodes/cosyvoice3/matcha/models/__init__.py diff --git a/cosyvoice3/matcha/models/components/__init__.py b/nodes/cosyvoice3/matcha/models/components/__init__.py similarity index 100% rename from cosyvoice3/matcha/models/components/__init__.py rename to nodes/cosyvoice3/matcha/models/components/__init__.py diff --git a/cosyvoice3/matcha/models/components/decoder.py b/nodes/cosyvoice3/matcha/models/components/decoder.py similarity index 100% rename from cosyvoice3/matcha/models/components/decoder.py rename to nodes/cosyvoice3/matcha/models/components/decoder.py diff --git a/cosyvoice3/matcha/models/components/flow_matching.py b/nodes/cosyvoice3/matcha/models/components/flow_matching.py similarity index 100% rename from cosyvoice3/matcha/models/components/flow_matching.py rename to nodes/cosyvoice3/matcha/models/components/flow_matching.py diff --git a/cosyvoice3/matcha/models/components/text_encoder.py b/nodes/cosyvoice3/matcha/models/components/text_encoder.py similarity index 100% rename from cosyvoice3/matcha/models/components/text_encoder.py rename to nodes/cosyvoice3/matcha/models/components/text_encoder.py diff --git a/cosyvoice3/matcha/models/components/transformer.py b/nodes/cosyvoice3/matcha/models/components/transformer.py similarity index 100% rename from cosyvoice3/matcha/models/components/transformer.py rename to nodes/cosyvoice3/matcha/models/components/transformer.py diff --git a/cosyvoice3/matcha/utils/__init__.py b/nodes/cosyvoice3/matcha/utils/__init__.py similarity index 100% rename from cosyvoice3/matcha/utils/__init__.py rename to nodes/cosyvoice3/matcha/utils/__init__.py diff --git a/cosyvoice3/matcha/utils/audio.py b/nodes/cosyvoice3/matcha/utils/audio.py similarity index 100% rename from cosyvoice3/matcha/utils/audio.py rename to nodes/cosyvoice3/matcha/utils/audio.py diff --git a/cosyvoice3/nodes/__init__.py b/nodes/cosyvoice3/nodes/__init__.py similarity index 100% rename from cosyvoice3/nodes/__init__.py rename to nodes/cosyvoice3/nodes/__init__.py diff --git a/cosyvoice3/nodes/audio_crop.py b/nodes/cosyvoice3/nodes/audio_crop.py similarity index 100% rename from cosyvoice3/nodes/audio_crop.py rename to nodes/cosyvoice3/nodes/audio_crop.py diff --git a/cosyvoice3/nodes/cross_lingual.py b/nodes/cosyvoice3/nodes/cross_lingual.py similarity index 100% rename from cosyvoice3/nodes/cross_lingual.py rename to nodes/cosyvoice3/nodes/cross_lingual.py diff --git a/cosyvoice3/nodes/dialog.py b/nodes/cosyvoice3/nodes/dialog.py similarity index 100% rename from cosyvoice3/nodes/dialog.py rename to nodes/cosyvoice3/nodes/dialog.py diff --git a/cosyvoice3/nodes/instruct2.py b/nodes/cosyvoice3/nodes/instruct2.py similarity index 100% rename from cosyvoice3/nodes/instruct2.py rename to nodes/cosyvoice3/nodes/instruct2.py diff --git a/cosyvoice3/nodes/model_loader.py b/nodes/cosyvoice3/nodes/model_loader.py similarity index 100% rename from cosyvoice3/nodes/model_loader.py rename to nodes/cosyvoice3/nodes/model_loader.py diff --git a/cosyvoice3/nodes/save_speaker.py b/nodes/cosyvoice3/nodes/save_speaker.py similarity index 100% rename from cosyvoice3/nodes/save_speaker.py rename to nodes/cosyvoice3/nodes/save_speaker.py diff --git a/cosyvoice3/nodes/speaker_clone.py b/nodes/cosyvoice3/nodes/speaker_clone.py similarity index 100% rename from cosyvoice3/nodes/speaker_clone.py rename to nodes/cosyvoice3/nodes/speaker_clone.py diff --git a/cosyvoice3/nodes/speaker_instruct2.py b/nodes/cosyvoice3/nodes/speaker_instruct2.py similarity index 100% rename from cosyvoice3/nodes/speaker_instruct2.py rename to nodes/cosyvoice3/nodes/speaker_instruct2.py diff --git a/cosyvoice3/nodes/voice_conversion.py b/nodes/cosyvoice3/nodes/voice_conversion.py similarity index 100% rename from cosyvoice3/nodes/voice_conversion.py rename to nodes/cosyvoice3/nodes/voice_conversion.py diff --git a/cosyvoice3/nodes/zero_shot.py b/nodes/cosyvoice3/nodes/zero_shot.py similarity index 100% rename from cosyvoice3/nodes/zero_shot.py rename to nodes/cosyvoice3/nodes/zero_shot.py diff --git a/cosyvoice3/utils/__init__.py b/nodes/cosyvoice3/utils/__init__.py similarity index 100% rename from cosyvoice3/utils/__init__.py rename to nodes/cosyvoice3/utils/__init__.py diff --git a/cosyvoice3/utils/audio_utils.py b/nodes/cosyvoice3/utils/audio_utils.py similarity index 100% rename from cosyvoice3/utils/audio_utils.py rename to nodes/cosyvoice3/utils/audio_utils.py diff --git a/cosyvoice3/utils/model_manager.py b/nodes/cosyvoice3/utils/model_manager.py similarity index 100% rename from cosyvoice3/utils/model_manager.py rename to nodes/cosyvoice3/utils/model_manager.py diff --git a/cosyvoice3/web/appearance.js b/nodes/cosyvoice3/web/appearance.js similarity index 100% rename from cosyvoice3/web/appearance.js rename to nodes/cosyvoice3/web/appearance.js diff --git a/cosyvoice3/workflows/CosyVoice.json b/nodes/cosyvoice3/workflows/CosyVoice.json similarity index 100% rename from cosyvoice3/workflows/CosyVoice.json rename to nodes/cosyvoice3/workflows/CosyVoice.json diff --git a/nodes/hardware/__init__.py b/nodes/hardware/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/nodes_memory_viz.py b/nodes/hardware/nodes_memory_viz.py similarity index 99% rename from nodes_memory_viz.py rename to nodes/hardware/nodes_memory_viz.py index b3abc5a..59bd1fc 100644 --- a/nodes_memory_viz.py +++ b/nodes/hardware/nodes_memory_viz.py @@ -67,7 +67,7 @@ def _init_lhm(): print(f"[XB 硬件监控] [WARN] clr 模块加载失败: {e}") return False - dll_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), + dll_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "LibreHardwareMonitorLib.dll") if not os.path.exists(dll_path): print(f"[XB 硬件监控] [WARN] LibreHardwareMonitorLib.dll 未找到: {dll_path}") diff --git a/nodes_vis.py b/nodes/hardware/nodes_vis.py similarity index 98% rename from nodes_vis.py rename to nodes/hardware/nodes_vis.py index 08ddd32..39e8a89 100644 --- a/nodes_vis.py +++ b/nodes/hardware/nodes_vis.py @@ -1,316 +1,316 @@ -import torch -import numpy as np -import os -from PIL import Image, ImageDraw, ImageFont - -class CalculatedFloat(float): - pass - -# ============================================================ -# XB_VRAM_Calculator — 显存计算器 -# ============================================================ -class XB_VRAM_Calculator: - def __init__(self): - pass - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "Total_VRAM_GB": ("INT", {"default": 24, "min": 4, "max": 128, "step": 1, "display": "number"}), - "System_Overhead_GB": ("FLOAT", {"default": 2.0, "min": 0.0, "max": 64.0, "step": 0.5, "display": "number"}), - "Main_Video_Model": (["LTX-2.3 (22B)", "WAN2.2-I2V (14B)", "WAN2.2-T2V (14B)", "WAN2.2-Animate (14B)"], {"default": "LTX-2.3 (22B)"}), - "Model_Quantization": (["FP16/BF16", "FP8", "GGUF-Q8", "GGUF-Q6", "GGUF-Q4"], {"default": "FP8"}), - "Total_Model_Layers": ("INT", {"default": 48, "min": 1, "max": 200, "step": 1, "display": "number"}), - "Layers_to_Swap": ("INT", {"default": 0, "min": 0, "max": 200, "step": 1, "display": "number"}), - "LoRA_Plugin_VRAM_GB": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 64.0, "step": 0.1, "display": "number"}), - } - } - - RETURN_TYPES = ("FLOAT",) - RETURN_NAMES = ("Available_VRAM",) - FUNCTION = "calculate_vram" - CATEGORY = "XB_ToolBox/Chunk_Preview" - - def calculate_vram(self, Total_VRAM_GB, System_Overhead_GB, Main_Video_Model, Model_Quantization, Total_Model_Layers, Layers_to_Swap, LoRA_Plugin_VRAM_GB): - vram_map = { - "LTX-2.3 (22B)": {"FP16/BF16": 44.0, "FP8": 22.5, "GGUF-Q8": 22.8, "GGUF-Q6": 17.8, "GGUF-Q4": 13.5}, - "WAN2.2-I2V (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, - "WAN2.2-T2V (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, - "WAN2.2-Animate (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, - } - - full_model_size = vram_map.get(Main_Video_Model, {}).get(Model_Quantization, 14.0) - stay_layers = max(0, Total_Model_Layers - Layers_to_Swap) - active_model_vram = (full_model_size / max(1, Total_Model_Layers)) * stay_layers - available_vram = Total_VRAM_GB - System_Overhead_GB - active_model_vram - LoRA_Plugin_VRAM_GB - available_vram = max(0.1, available_vram) - - res = CalculatedFloat(available_vram) - res._stay = stay_layers - res._total = Total_Model_Layers - - return (res,) - -# ============================================================ -# XB_ChunkVisualization — 分块可视化 -# ============================================================ -class XB_ChunkVisualization: - def __init__(self): - pass - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "Stat_Mode": (["Spatial Only", "Temporal Only", "Spatial & Temporal"], {"default": "Spatial & Temporal"}), - "Available_VRAM_GB": ("FLOAT", {"default": 12.0, "min": 0.1, "max": 128.0, "step": 0.1, "display": "number"}), - "Image_Width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16, "display": "number"}), - "Image_Height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16, "display": "number"}), - "Image_Frames": ("INT", {"default": 1, "min": 1, "max": 10000, "step": 4, "display": "number"}), - "Current_Stage": (["Encode Stage", "Decode Stage"], {"default": "Decode Stage"}), - "Spatial_Tile_Size": ("INT", {"default": 512, "min": 64, "max": 8192, "step": 32, "display": "number"}), - "Spatial_Tile_Overlap": ("INT", {"default": 64, "min": 0, "max": 4096, "step": 32, "display": "number"}), - "Temporal_Chunk_Size": ("INT", {"default": 33, "min": 8, "max": 10000, "step": 4, "display": "number"}), - "Temporal_Chunk_Overlap": ("INT", {"default": 4, "min": 0, "max": 1000, "step": 4, "display": "number"}), - }, - "optional": { - "Image_Input": ("IMAGE",) - } - } - - RETURN_TYPES = ("IMAGE",) - RETURN_NAMES = ("preview_image",) - FUNCTION = "visualize_chunks" - CATEGORY = "XB_ToolBox" - - def get_1d_chunks(self, total_len, chunk_size, overlap): - chunks = [] - if chunk_size <= 0: return [(0, total_len)] - stride = max(1, chunk_size - min(overlap, int(chunk_size * 0.9))) - curr = 0 - while curr < total_len: - start, end = curr, min(curr + chunk_size, total_len) - chunks.append((start, end)) - if end >= total_len: break - curr += stride - return chunks - - def visualize_chunks(self, Stat_Mode, Available_VRAM_GB, Image_Width, Image_Height, Image_Frames, Current_Stage, - Spatial_Tile_Size, Spatial_Tile_Overlap, Temporal_Chunk_Size, Temporal_Chunk_Overlap, Image_Input=None): - - active_vram = float(Available_VRAM_GB) - is_calc = type(Available_VRAM_GB).__name__ == "CalculatedFloat" - stay_info = f" (Reside:{Available_VRAM_GB._stay}/{Available_VRAM_GB._total}L)" if (is_calc and hasattr(Available_VRAM_GB, '_stay')) else "" - - if Image_Input is not None: - B, H_orig, W_orig, C = Image_Input.shape - source_img = Image.fromarray((Image_Input[0].cpu().numpy() * 255).astype(np.uint8)).convert("RGBA") - res_info = f"[Size]: {W_orig}x{H_orig} (Input)" - else: - W_orig, H_orig = Image_Width, Image_Height - source_img = Image.new("RGBA", (W_orig, H_orig), (25, 25, 25, 255)) - res_info = f"[Size]: {W_orig}x{H_orig} (Param)" - - x_chunks = self.get_1d_chunks(W_orig, Spatial_Tile_Size if "Spatial" in Stat_Mode else 0, Spatial_Tile_Overlap) - y_chunks = self.get_1d_chunks(H_orig, Spatial_Tile_Size if "Spatial" in Stat_Mode else 0, Spatial_Tile_Overlap) - t_chunks = self.get_1d_chunks(Image_Frames, Temporal_Chunk_Size if "Temporal" in Stat_Mode else 0, Temporal_Chunk_Overlap) - total_batches = len(x_chunks) * len(y_chunks) * len(t_chunks) - - base_vol = W_orig * H_orig * Image_Frames - real_vol, max_chunk_vol = 0, 0 - for x1, x2 in x_chunks: - for y1, y2 in y_chunks: - for t1, t2 in t_chunks: - v = (x2-x1)*(y2-y1)*(t2-t1) - real_vol += v - max_chunk_vol = max(max_chunk_vol, v) - - redundancy = ((real_vol - base_vol) / base_vol * 100) if base_vol > 0 else 0 - - safe_unit = active_vram * 5_000_000 - kappa = (max_chunk_vol / max(1, safe_unit)) ** 1.15 - p_percent = min(100, (kappa / 2.0) * 100) - p_color = (0, 255, 0) if p_percent < 60 else ((255, 200, 0) if p_percent < 85 else (255, 50, 50)) - - # 🛡️ 全局字体缓存：避免 Auto-Queue 模式下反复磁盘 I/O 和内存泄漏 - _FONT_CACHE = {} - - def get_font(size): - font_paths = [ - # Windows - os.path.join(os.environ.get('WINDIR', 'C:/Windows'), "Fonts", "msyh.ttc"), - os.path.join(os.environ.get('WINDIR', 'C:/Windows'), "Fonts", "simhei.ttf"), - # Linux - "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", - "/usr/share/fonts/truetype/wqy/wqy-microhei.ttc", - # macOS - "/System/Library/Fonts/STHeiti Medium.ttc", - "/System/Library/Fonts/Helvetica.ttc", - ] - for p in font_paths: - cache_key = (p, size) - if cache_key in _FONT_CACHE: - return _FONT_CACHE[cache_key] - if os.path.exists(p): - try: - font = ImageFont.truetype(p, size) - _FONT_CACHE[cache_key] = font - return font - except Exception: - continue - # 兜底字体也缓存 - if ("_fallback", size) in _FONT_CACHE: - return _FONT_CACHE[("_fallback", size)] - fb = ImageFont.load_default() - _FONT_CACHE[("_fallback", size)] = fb - return fb - - f_info = get_font(22) - f_num = get_font(28) - temp_draw = ImageDraw.Draw(Image.new("RGBA", (1,1))) - - show_time = ("Temporal" in Stat_Mode and Image_Frames > 1) - sidebar_w = 250 if show_time else 0 - gap = 50 if show_time else 0 - padding = 70 - - draw_W = W_orig + gap + sidebar_w - new_W = max(draw_W + 80, 700) - max_txt_w = new_W - 50 - - info_cats = [ - {"text": f"[Mode]: {Stat_Mode} / [Stage]: {Current_Stage}", "color": (0, 255, 255)}, - {"text": res_info, "color": (220, 220, 220)}, - {"text": f"[Batches]: {total_batches} (S:{len(x_chunks)*len(y_chunks)} x T:{len(t_chunks)})", "color": (255, 255, 255)}, - {"text": f"[VRAM]: {active_vram:.1f}G{stay_info}", "color": (150, 200, 255)}, - {"text": f"[Volume]: {real_vol/1e6:.1f}M (Base:{base_vol/1e6:.1f}M | Redundancy:+{redundancy:.1f}%)", "color": (255, 150, 200)}, - {"text": f"[Chunk]: {max_chunk_vol/1e6:.1f}M/b / [Pressure]: {kappa:.2f}", "color": p_color} - ] - - final_lines = [] - for cat in info_cats: - prefix, content = cat["text"].split("]: ", 1) if "]: " in cat["text"] else ("", cat["text"]) - if prefix: prefix += "]: " - prefix_w = int(temp_draw.textlength(prefix, font=f_info)) - curr, is_first = "", True - for char in content: - limit = max_txt_w if is_first else (max_txt_w - prefix_w) - if temp_draw.textlength(curr + char, font=f_info) <= limit: - curr += char - else: - final_lines.append((prefix + curr if is_first else curr, cat["color"], 0 if is_first else prefix_w)) - curr, is_first = char, False - if curr: - final_lines.append((prefix + curr if is_first else curr, cat["color"], 0 if is_first else prefix_w)) - - line_h = 36 - header_H = len(final_lines) * line_h + 80 - - paste_x = int((new_W - draw_W) / 2) - paste_y = header_H + padding - - full_canvas = Image.new("RGBA", (int(new_W), int(paste_y + H_orig + 40)), (20, 20, 20, 255)) - - if Stat_Mode != "Temporal Only": - full_canvas.paste(source_img, (paste_x, paste_y)) - else: - faded = source_img.copy() - faded.putalpha(60) - full_canvas.paste(faded, (paste_x, paste_y), faded) - - draw = ImageDraw.Draw(full_canvas, "RGBA") - - y = 15 - for txt, clr, x_off in final_lines: - draw.text((25 + x_off, y), txt, fill=clr, font=f_info) - y += line_h - - draw.rounded_rectangle([25, y+10, 525, y+22], radius=6, fill=(45, 45, 45)) - draw.rounded_rectangle([25, y+10, 25 + int(500 * (p_percent/100)), y+22], radius=6, fill=p_color) - - colors = [(0, 255, 0), (0, 255, 255), (255, 255, 0), (255, 0, 255), (0, 150, 255), (255, 120, 0), (255, 50, 50)] - - title_y = paste_y - 45 - bar_cx = paste_x + W_orig + gap + sidebar_w // 2 - 40 - if "Spatial" in Stat_Mode: - draw.text((paste_x + W_orig//2 - 84, title_y), "[Spatial]", fill=(220,220,220,255), font=f_num) - if show_time: - draw.text((bar_cx - 84, title_y), "[Temporal]", fill=(220,220,220,255), font=f_num) - - if "Spatial" in Stat_Mode: - spatial_idx = 0 - for y1, y2 in y_chunks: - for x1, x2 in x_chunks: - c = colors[spatial_idx % len(colors)] - rect = [paste_x + x1, paste_y + y1, paste_x + x2, paste_y + y2] - draw.rectangle(rect, fill=None, outline=c, width=3) - - if total_batches < 150: - lbl_x, lbl_y = paste_x + x1 + 10, paste_y + y1 + 10 - draw.text((lbl_x, lbl_y), f"S{spatial_idx+1}", fill=c, font=f_num, stroke_width=3, stroke_fill=(0,0,0,255)) - - spatial_idx += 1 - - if show_time: - rx = 60 - ry = 18 - - draw.line([bar_cx, paste_y, bar_cx, paste_y + H_orig], fill=(70, 70, 70, 255), width=2) - scale_y = H_orig / max(1, Image_Frames) - - def draw_cylinder(d, cx, cy1, cy2, rx, ry, fill_color, outline_color, line_width): - if fill_color: - d.ellipse([cx - rx, cy2 - ry, cx + rx, cy2 + ry], fill=fill_color, outline=outline_color, width=line_width) - d.rectangle([cx - rx, cy1, cx + rx, cy2], fill=fill_color) - d.line([cx - rx, cy1, cx - rx, cy2], fill=outline_color, width=line_width) - d.line([cx + rx, cy1, cx + rx, cy2], fill=outline_color, width=line_width) - d.ellipse([cx - rx, cy1 - ry, cx + rx, cy1 + ry], fill=fill_color, outline=outline_color, width=line_width) - else: - d.ellipse([cx - rx, cy2 - ry, cx + rx, cy2 + ry], fill=None, outline=outline_color, width=line_width) - d.line([cx - rx, cy1, cx - rx, cy2], fill=outline_color, width=line_width) - d.line([cx + rx, cy1, cx + rx, cy2], fill=outline_color, width=line_width) - d.ellipse([cx - rx, cy1 - ry, cx + rx, cy1 + ry], fill=None, outline=outline_color, width=line_width) - - for i in range(len(t_chunks)-1, -1, -1): - t1, t2 = t_chunks[i] - y1 = paste_y + int(t1 * scale_y) - y2 = paste_y + int(t2 * scale_y) - - true_mid_y = (y1 + y2) / 2 - if y2 - y1 < ry * 2: - y1 = true_mid_y - ry - y2 = true_mid_y + ry - - c = colors[i % len(colors)] - body_color = (c[0]//5, c[1]//5, c[2]//5, 200) - draw_cylinder(draw, bar_cx, y1, y2, rx, ry, fill_color=body_color, outline_color=c, line_width=2) - - draw.line([bar_cx + rx + 5, true_mid_y, bar_cx + rx + 15, true_mid_y], fill=c, width=3) - draw.text((bar_cx + rx + 22, true_mid_y - 15), f"T{i+1}: F{t1}-{t2}", fill=c, font=f_info, stroke_width=3, stroke_fill=(0,0,0,255)) - - overlaps = [] - for i in range(len(t_chunks)): - for j in range(i+1, len(t_chunks)): - a, b = t_chunks[i] - c, d = t_chunks[j] - o_start, o_end = max(a, c), min(b, d) - if o_start < o_end: - overlaps.append((o_start, o_end)) - - for o_start, o_end in overlaps: - y1 = paste_y + int(o_start * scale_y) - y2 = paste_y + int(o_end * scale_y) - - true_mid_y = (y1 + y2) / 2 - if y2 - y1 < ry * 2: - y1 = true_mid_y - ry - y2 = true_mid_y + ry - - draw_cylinder(draw, bar_cx, y1, y2, rx, ry, fill_color=None, outline_color=(255, 255, 255, 255), line_width=4) - - hl_color = (255, 255, 0, 255) - draw.line([bar_cx + rx + 4, true_mid_y, bar_cx + rx + 30, true_mid_y], fill=hl_color, width=3) - draw.text((bar_cx + rx + 35, true_mid_y - 15), "Overlap", fill=hl_color, font=f_info, stroke_width=2, stroke_fill=(0,0,0,255)) - +import torch +import numpy as np +import os +from PIL import Image, ImageDraw, ImageFont + +class CalculatedFloat(float): + pass + +# ============================================================ +# XB_VRAM_Calculator — 显存计算器 +# ============================================================ +class XB_VRAM_Calculator: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "Total_VRAM_GB": ("INT", {"default": 24, "min": 4, "max": 128, "step": 1, "display": "number"}), + "System_Overhead_GB": ("FLOAT", {"default": 2.0, "min": 0.0, "max": 64.0, "step": 0.5, "display": "number"}), + "Main_Video_Model": (["LTX-2.3 (22B)", "WAN2.2-I2V (14B)", "WAN2.2-T2V (14B)", "WAN2.2-Animate (14B)"], {"default": "LTX-2.3 (22B)"}), + "Model_Quantization": (["FP16/BF16", "FP8", "GGUF-Q8", "GGUF-Q6", "GGUF-Q4"], {"default": "FP8"}), + "Total_Model_Layers": ("INT", {"default": 48, "min": 1, "max": 200, "step": 1, "display": "number"}), + "Layers_to_Swap": ("INT", {"default": 0, "min": 0, "max": 200, "step": 1, "display": "number"}), + "LoRA_Plugin_VRAM_GB": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 64.0, "step": 0.1, "display": "number"}), + } + } + + RETURN_TYPES = ("FLOAT",) + RETURN_NAMES = ("Available_VRAM",) + FUNCTION = "calculate_vram" + CATEGORY = "XB_ToolBox/Chunk_Preview" + + def calculate_vram(self, Total_VRAM_GB, System_Overhead_GB, Main_Video_Model, Model_Quantization, Total_Model_Layers, Layers_to_Swap, LoRA_Plugin_VRAM_GB): + vram_map = { + "LTX-2.3 (22B)": {"FP16/BF16": 44.0, "FP8": 22.5, "GGUF-Q8": 22.8, "GGUF-Q6": 17.8, "GGUF-Q4": 13.5}, + "WAN2.2-I2V (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, + "WAN2.2-T2V (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, + "WAN2.2-Animate (14B)": {"FP16/BF16": 28.0, "FP8": 14.0, "GGUF-Q8": 14.5, "GGUF-Q6": 11.0, "GGUF-Q4": 9.0}, + } + + full_model_size = vram_map.get(Main_Video_Model, {}).get(Model_Quantization, 14.0) + stay_layers = max(0, Total_Model_Layers - Layers_to_Swap) + active_model_vram = (full_model_size / max(1, Total_Model_Layers)) * stay_layers + available_vram = Total_VRAM_GB - System_Overhead_GB - active_model_vram - LoRA_Plugin_VRAM_GB + available_vram = max(0.1, available_vram) + + res = CalculatedFloat(available_vram) + res._stay = stay_layers + res._total = Total_Model_Layers + + return (res,) + +# ============================================================ +# XB_ChunkVisualization — 分块可视化 +# ============================================================ +class XB_ChunkVisualization: + def __init__(self): + pass + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "Stat_Mode": (["Spatial Only", "Temporal Only", "Spatial & Temporal"], {"default": "Spatial & Temporal"}), + "Available_VRAM_GB": ("FLOAT", {"default": 12.0, "min": 0.1, "max": 128.0, "step": 0.1, "display": "number"}), + "Image_Width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16, "display": "number"}), + "Image_Height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16, "display": "number"}), + "Image_Frames": ("INT", {"default": 1, "min": 1, "max": 10000, "step": 4, "display": "number"}), + "Current_Stage": (["Encode Stage", "Decode Stage"], {"default": "Decode Stage"}), + "Spatial_Tile_Size": ("INT", {"default": 512, "min": 64, "max": 8192, "step": 32, "display": "number"}), + "Spatial_Tile_Overlap": ("INT", {"default": 64, "min": 0, "max": 4096, "step": 32, "display": "number"}), + "Temporal_Chunk_Size": ("INT", {"default": 33, "min": 8, "max": 10000, "step": 4, "display": "number"}), + "Temporal_Chunk_Overlap": ("INT", {"default": 4, "min": 0, "max": 1000, "step": 4, "display": "number"}), + }, + "optional": { + "Image_Input": ("IMAGE",) + } + } + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("preview_image",) + FUNCTION = "visualize_chunks" + CATEGORY = "XB_ToolBox" + + def get_1d_chunks(self, total_len, chunk_size, overlap): + chunks = [] + if chunk_size <= 0: return [(0, total_len)] + stride = max(1, chunk_size - min(overlap, int(chunk_size * 0.9))) + curr = 0 + while curr < total_len: + start, end = curr, min(curr + chunk_size, total_len) + chunks.append((start, end)) + if end >= total_len: break + curr += stride + return chunks + + def visualize_chunks(self, Stat_Mode, Available_VRAM_GB, Image_Width, Image_Height, Image_Frames, Current_Stage, + Spatial_Tile_Size, Spatial_Tile_Overlap, Temporal_Chunk_Size, Temporal_Chunk_Overlap, Image_Input=None): + + active_vram = float(Available_VRAM_GB) + is_calc = type(Available_VRAM_GB).__name__ == "CalculatedFloat" + stay_info = f" (Reside:{Available_VRAM_GB._stay}/{Available_VRAM_GB._total}L)" if (is_calc and hasattr(Available_VRAM_GB, '_stay')) else "" + + if Image_Input is not None: + B, H_orig, W_orig, C = Image_Input.shape + source_img = Image.fromarray((Image_Input[0].cpu().numpy() * 255).astype(np.uint8)).convert("RGBA") + res_info = f"[Size]: {W_orig}x{H_orig} (Input)" + else: + W_orig, H_orig = Image_Width, Image_Height + source_img = Image.new("RGBA", (W_orig, H_orig), (25, 25, 25, 255)) + res_info = f"[Size]: {W_orig}x{H_orig} (Param)" + + x_chunks = self.get_1d_chunks(W_orig, Spatial_Tile_Size if "Spatial" in Stat_Mode else 0, Spatial_Tile_Overlap) + y_chunks = self.get_1d_chunks(H_orig, Spatial_Tile_Size if "Spatial" in Stat_Mode else 0, Spatial_Tile_Overlap) + t_chunks = self.get_1d_chunks(Image_Frames, Temporal_Chunk_Size if "Temporal" in Stat_Mode else 0, Temporal_Chunk_Overlap) + total_batches = len(x_chunks) * len(y_chunks) * len(t_chunks) + + base_vol = W_orig * H_orig * Image_Frames + real_vol, max_chunk_vol = 0, 0 + for x1, x2 in x_chunks: + for y1, y2 in y_chunks: + for t1, t2 in t_chunks: + v = (x2-x1)*(y2-y1)*(t2-t1) + real_vol += v + max_chunk_vol = max(max_chunk_vol, v) + + redundancy = ((real_vol - base_vol) / base_vol * 100) if base_vol > 0 else 0 + + safe_unit = active_vram * 5_000_000 + kappa = (max_chunk_vol / max(1, safe_unit)) ** 1.15 + p_percent = min(100, (kappa / 2.0) * 100) + p_color = (0, 255, 0) if p_percent < 60 else ((255, 200, 0) if p_percent < 85 else (255, 50, 50)) + + # 🛡️ 全局字体缓存：避免 Auto-Queue 模式下反复磁盘 I/O 和内存泄漏 + _FONT_CACHE = {} + + def get_font(size): + font_paths = [ + # Windows + os.path.join(os.environ.get('WINDIR', 'C:/Windows'), "Fonts", "msyh.ttc"), + os.path.join(os.environ.get('WINDIR', 'C:/Windows'), "Fonts", "simhei.ttf"), + # Linux + "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", + "/usr/share/fonts/truetype/wqy/wqy-microhei.ttc", + # macOS + "/System/Library/Fonts/STHeiti Medium.ttc", + "/System/Library/Fonts/Helvetica.ttc", + ] + for p in font_paths: + cache_key = (p, size) + if cache_key in _FONT_CACHE: + return _FONT_CACHE[cache_key] + if os.path.exists(p): + try: + font = ImageFont.truetype(p, size) + _FONT_CACHE[cache_key] = font + return font + except Exception: + continue + # 兜底字体也缓存 + if ("_fallback", size) in _FONT_CACHE: + return _FONT_CACHE[("_fallback", size)] + fb = ImageFont.load_default() + _FONT_CACHE[("_fallback", size)] = fb + return fb + + f_info = get_font(22) + f_num = get_font(28) + temp_draw = ImageDraw.Draw(Image.new("RGBA", (1,1))) + + show_time = ("Temporal" in Stat_Mode and Image_Frames > 1) + sidebar_w = 250 if show_time else 0 + gap = 50 if show_time else 0 + padding = 70 + + draw_W = W_orig + gap + sidebar_w + new_W = max(draw_W + 80, 700) + max_txt_w = new_W - 50 + + info_cats = [ + {"text": f"[Mode]: {Stat_Mode} / [Stage]: {Current_Stage}", "color": (0, 255, 255)}, + {"text": res_info, "color": (220, 220, 220)}, + {"text": f"[Batches]: {total_batches} (S:{len(x_chunks)*len(y_chunks)} x T:{len(t_chunks)})", "color": (255, 255, 255)}, + {"text": f"[VRAM]: {active_vram:.1f}G{stay_info}", "color": (150, 200, 255)}, + {"text": f"[Volume]: {real_vol/1e6:.1f}M (Base:{base_vol/1e6:.1f}M | Redundancy:+{redundancy:.1f}%)", "color": (255, 150, 200)}, + {"text": f"[Chunk]: {max_chunk_vol/1e6:.1f}M/b / [Pressure]: {kappa:.2f}", "color": p_color} + ] + + final_lines = [] + for cat in info_cats: + prefix, content = cat["text"].split("]: ", 1) if "]: " in cat["text"] else ("", cat["text"]) + if prefix: prefix += "]: " + prefix_w = int(temp_draw.textlength(prefix, font=f_info)) + curr, is_first = "", True + for char in content: + limit = max_txt_w if is_first else (max_txt_w - prefix_w) + if temp_draw.textlength(curr + char, font=f_info) <= limit: + curr += char + else: + final_lines.append((prefix + curr if is_first else curr, cat["color"], 0 if is_first else prefix_w)) + curr, is_first = char, False + if curr: + final_lines.append((prefix + curr if is_first else curr, cat["color"], 0 if is_first else prefix_w)) + + line_h = 36 + header_H = len(final_lines) * line_h + 80 + + paste_x = int((new_W - draw_W) / 2) + paste_y = header_H + padding + + full_canvas = Image.new("RGBA", (int(new_W), int(paste_y + H_orig + 40)), (20, 20, 20, 255)) + + if Stat_Mode != "Temporal Only": + full_canvas.paste(source_img, (paste_x, paste_y)) + else: + faded = source_img.copy() + faded.putalpha(60) + full_canvas.paste(faded, (paste_x, paste_y), faded) + + draw = ImageDraw.Draw(full_canvas, "RGBA") + + y = 15 + for txt, clr, x_off in final_lines: + draw.text((25 + x_off, y), txt, fill=clr, font=f_info) + y += line_h + + draw.rounded_rectangle([25, y+10, 525, y+22], radius=6, fill=(45, 45, 45)) + draw.rounded_rectangle([25, y+10, 25 + int(500 * (p_percent/100)), y+22], radius=6, fill=p_color) + + colors = [(0, 255, 0), (0, 255, 255), (255, 255, 0), (255, 0, 255), (0, 150, 255), (255, 120, 0), (255, 50, 50)] + + title_y = paste_y - 45 + bar_cx = paste_x + W_orig + gap + sidebar_w // 2 - 40 + if "Spatial" in Stat_Mode: + draw.text((paste_x + W_orig//2 - 84, title_y), "[Spatial]", fill=(220,220,220,255), font=f_num) + if show_time: + draw.text((bar_cx - 84, title_y), "[Temporal]", fill=(220,220,220,255), font=f_num) + + if "Spatial" in Stat_Mode: + spatial_idx = 0 + for y1, y2 in y_chunks: + for x1, x2 in x_chunks: + c = colors[spatial_idx % len(colors)] + rect = [paste_x + x1, paste_y + y1, paste_x + x2, paste_y + y2] + draw.rectangle(rect, fill=None, outline=c, width=3) + + if total_batches < 150: + lbl_x, lbl_y = paste_x + x1 + 10, paste_y + y1 + 10 + draw.text((lbl_x, lbl_y), f"S{spatial_idx+1}", fill=c, font=f_num, stroke_width=3, stroke_fill=(0,0,0,255)) + + spatial_idx += 1 + + if show_time: + rx = 60 + ry = 18 + + draw.line([bar_cx, paste_y, bar_cx, paste_y + H_orig], fill=(70, 70, 70, 255), width=2) + scale_y = H_orig / max(1, Image_Frames) + + def draw_cylinder(d, cx, cy1, cy2, rx, ry, fill_color, outline_color, line_width): + if fill_color: + d.ellipse([cx - rx, cy2 - ry, cx + rx, cy2 + ry], fill=fill_color, outline=outline_color, width=line_width) + d.rectangle([cx - rx, cy1, cx + rx, cy2], fill=fill_color) + d.line([cx - rx, cy1, cx - rx, cy2], fill=outline_color, width=line_width) + d.line([cx + rx, cy1, cx + rx, cy2], fill=outline_color, width=line_width) + d.ellipse([cx - rx, cy1 - ry, cx + rx, cy1 + ry], fill=fill_color, outline=outline_color, width=line_width) + else: + d.ellipse([cx - rx, cy2 - ry, cx + rx, cy2 + ry], fill=None, outline=outline_color, width=line_width) + d.line([cx - rx, cy1, cx - rx, cy2], fill=outline_color, width=line_width) + d.line([cx + rx, cy1, cx + rx, cy2], fill=outline_color, width=line_width) + d.ellipse([cx - rx, cy1 - ry, cx + rx, cy1 + ry], fill=None, outline=outline_color, width=line_width) + + for i in range(len(t_chunks)-1, -1, -1): + t1, t2 = t_chunks[i] + y1 = paste_y + int(t1 * scale_y) + y2 = paste_y + int(t2 * scale_y) + + true_mid_y = (y1 + y2) / 2 + if y2 - y1 < ry * 2: + y1 = true_mid_y - ry + y2 = true_mid_y + ry + + c = colors[i % len(colors)] + body_color = (c[0]//5, c[1]//5, c[2]//5, 200) + draw_cylinder(draw, bar_cx, y1, y2, rx, ry, fill_color=body_color, outline_color=c, line_width=2) + + draw.line([bar_cx + rx + 5, true_mid_y, bar_cx + rx + 15, true_mid_y], fill=c, width=3) + draw.text((bar_cx + rx + 22, true_mid_y - 15), f"T{i+1}: F{t1}-{t2}", fill=c, font=f_info, stroke_width=3, stroke_fill=(0,0,0,255)) + + overlaps = [] + for i in range(len(t_chunks)): + for j in range(i+1, len(t_chunks)): + a, b = t_chunks[i] + c, d = t_chunks[j] + o_start, o_end = max(a, c), min(b, d) + if o_start < o_end: + overlaps.append((o_start, o_end)) + + for o_start, o_end in overlaps: + y1 = paste_y + int(o_start * scale_y) + y2 = paste_y + int(o_end * scale_y) + + true_mid_y = (y1 + y2) / 2 + if y2 - y1 < ry * 2: + y1 = true_mid_y - ry + y2 = true_mid_y + ry + + draw_cylinder(draw, bar_cx, y1, y2, rx, ry, fill_color=None, outline_color=(255, 255, 255, 255), line_width=4) + + hl_color = (255, 255, 0, 255) + draw.line([bar_cx + rx + 4, true_mid_y, bar_cx + rx + 30, true_mid_y], fill=hl_color, width=3) + draw.text((bar_cx + rx + 35, true_mid_y - 15), "Overlap", fill=hl_color, font=f_info, stroke_width=2, stroke_fill=(0,0,0,255)) + return (torch.from_numpy(np.array(full_canvas.convert("RGB")).astype(np.float32) / 255.0).unsqueeze(0),) \ No newline at end of file diff --git a/nodes_vram.py b/nodes/hardware/nodes_vram.py similarity index 97% rename from nodes_vram.py rename to nodes/hardware/nodes_vram.py index 039d13f..e0b397b 100644 --- a/nodes_vram.py +++ b/nodes/hardware/nodes_vram.py @@ -1,67 +1,67 @@ -import torch -import gc -import comfy.model_management as mm - -# ============================================================ -# XTX_Data_Radar — 数据雷达扫描 -# ============================================================ -class XTX_Data_Radar: - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "model": ("MODEL",), - "latent": ("LATENT",), - }, - "optional": { - "positive_cond": ("CONDITIONING",), - "negative_cond": ("CONDITIONING",), - } - } - - RETURN_TYPES = ("MODEL", "LATENT") - RETURN_NAMES = ("model", "latent") - FUNCTION = "scan_data" - CATEGORY = "XB_ToolBox/VRAM_Hacks" - - def scan_data(self, model, latent, positive_cond=None, negative_cond=None): - print("\n" + "=" * 35) - print(f"🪞 [XB-BOX] Data Generation Radar Scan 🔍") - print("-" * 35) - - if "samples" in latent: - samples = latent["samples"] - size_mb = (samples.element_size() * samples.nelement()) / (1024 * 1024) - print(f"📦 Latent: {size_mb:.2f} MB") - print(f"📐 Shape: {list(samples.shape)}") - - def get_cond_weight(cond): - total_size = 0 - if cond is not None: - for c in cond: - if isinstance(c, list) and len(c) > 0: - # c[0]: 交叉注意力条件张量 - tensor = c[0] - if isinstance(tensor, torch.Tensor): - total_size += tensor.element_size() * tensor.nelement() - # c[1]: 字典，含 pooled_output / ControlNet / GLIGEN 等重型特征 - if len(c) > 1 and isinstance(c[1], dict): - for _k, _v in c[1].items(): - if isinstance(_v, torch.Tensor): - total_size += _v.element_size() * _v.nelement() - return total_size / (1024 * 1024) - - pos_w = get_cond_weight(positive_cond) - neg_w = get_cond_weight(negative_cond) - print(f"✅ Positive Cond Weight: {pos_w:.2f} MB") - print(f"❌ Negative Cond Weight: {neg_w:.2f} MB") - print("-" * 35) - - if torch.cuda.is_available(): - allocated = torch.cuda.memory_allocated() / (1024**3) - reserved = torch.cuda.memory_reserved() / (1024**3) - print(f"🔥 GPU Current Allocated: {allocated:.2f} GB") - print(f"🛡️ GPU Current Reserved: {reserved:.2f} GB") - print("=" * 35 + "\n") - +import torch +import gc +import comfy.model_management as mm + +# ============================================================ +# XTX_Data_Radar — 数据雷达扫描 +# ============================================================ +class XTX_Data_Radar: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "model": ("MODEL",), + "latent": ("LATENT",), + }, + "optional": { + "positive_cond": ("CONDITIONING",), + "negative_cond": ("CONDITIONING",), + } + } + + RETURN_TYPES = ("MODEL", "LATENT") + RETURN_NAMES = ("model", "latent") + FUNCTION = "scan_data" + CATEGORY = "XB_ToolBox/VRAM_Hacks" + + def scan_data(self, model, latent, positive_cond=None, negative_cond=None): + print("\n" + "=" * 35) + print(f"🪞 [XB-BOX] Data Generation Radar Scan 🔍") + print("-" * 35) + + if "samples" in latent: + samples = latent["samples"] + size_mb = (samples.element_size() * samples.nelement()) / (1024 * 1024) + print(f"📦 Latent: {size_mb:.2f} MB") + print(f"📐 Shape: {list(samples.shape)}") + + def get_cond_weight(cond): + total_size = 0 + if cond is not None: + for c in cond: + if isinstance(c, list) and len(c) > 0: + # c[0]: 交叉注意力条件张量 + tensor = c[0] + if isinstance(tensor, torch.Tensor): + total_size += tensor.element_size() * tensor.nelement() + # c[1]: 字典，含 pooled_output / ControlNet / GLIGEN 等重型特征 + if len(c) > 1 and isinstance(c[1], dict): + for _k, _v in c[1].items(): + if isinstance(_v, torch.Tensor): + total_size += _v.element_size() * _v.nelement() + return total_size / (1024 * 1024) + + pos_w = get_cond_weight(positive_cond) + neg_w = get_cond_weight(negative_cond) + print(f"✅ Positive Cond Weight: {pos_w:.2f} MB") + print(f"❌ Negative Cond Weight: {neg_w:.2f} MB") + print("-" * 35) + + if torch.cuda.is_available(): + allocated = torch.cuda.memory_allocated() / (1024**3) + reserved = torch.cuda.memory_reserved() / (1024**3) + print(f"🔥 GPU Current Allocated: {allocated:.2f} GB") + print(f"🛡️ GPU Current Reserved: {reserved:.2f} GB") + print("=" * 35 + "\n") + return (model, latent) \ No newline at end of file diff --git a/nodes/image/__init__.py b/nodes/image/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/nodes_comic.py b/nodes/image/nodes_comic.py similarity index 100% rename from nodes_comic.py rename to nodes/image/nodes_comic.py diff --git a/nodes_flux_encode.py b/nodes/image/nodes_flux_encode.py similarity index 97% rename from nodes_flux_encode.py rename to nodes/image/nodes_flux_encode.py index 65a6187..39f1e46 100644 --- a/nodes_flux_encode.py +++ b/nodes/image/nodes_flux_encode.py @@ -8,7 +8,7 @@ XB-ToolBox FLUX多图编码节点 import node_helpers import nodes from comfy_extras.nodes_dataset import ResizeImagesByLongerEdgeNode -from .nodes_reference_any import FlexibleOptionalInputType +from ..core.nodes_reference_any import FlexibleOptionalInputType class XB_FluxMultiImageEncode: diff --git a/nodes_image_preset.py b/nodes/image/nodes_image_preset.py similarity index 99% rename from nodes_image_preset.py rename to nodes/image/nodes_image_preset.py index 67581bc..64468a9 100644 --- a/nodes_image_preset.py +++ b/nodes/image/nodes_image_preset.py @@ -10,7 +10,7 @@ （带素材清单的整套图库或普通图片文件夹都支持，见 xb_gallery.py）。 """ -from . import xb_gallery +from ...libs import xb_gallery from .nodes_image_prompt_preset import XB_ImagePromptPreset _REF_TOOLTIP = ("内部字段（节点表面不显示）：由「参考图选择」按钮写入的图库相对路径 JSON；" diff --git a/nodes_image_prompt_preset.py b/nodes/image/nodes_image_prompt_preset.py similarity index 99% rename from nodes_image_prompt_preset.py rename to nodes/image/nodes_image_prompt_preset.py index b7f9228..2709d6d 100644 --- a/nodes_image_prompt_preset.py +++ b/nodes/image/nodes_image_prompt_preset.py @@ -49,8 +49,8 @@ import torch import comfy.model_management import nodes -from .xb_skills import SKILL_NONE, skill_text -from . import xb_presets +from ...libs.xb_skills import SKILL_NONE, skill_text +from ...libs import xb_presets # ── 画幅比例（与本包 XB_ImageParamsMaster 完全一致的选项顺序） ────────────── ASPECT_RATIO_OPTIONS = ["Free", "1:1", "16:9", "9:16", "4:3", "3:4", "21:9", "2:3", "3:2"] diff --git a/nodes_image_prompt_preset_pro.py b/nodes/image/nodes_image_prompt_preset_pro.py similarity index 99% rename from nodes_image_prompt_preset_pro.py rename to nodes/image/nodes_image_prompt_preset_pro.py index b893f97..34c2d4b 100644 --- a/nodes_image_prompt_preset_pro.py +++ b/nodes/image/nodes_image_prompt_preset_pro.py @@ -38,13 +38,13 @@ import nodes import torch import comfy.model_management -from .nodes_llama import ( +from ..llm.nodes_llama import ( LLAMA_CPP_STORAGE, XB_llamaInstruct, XB_llamaPromptEnhancer, preset_tags, ) -from .nodes_llama_prompt_reverse import ( +from ..llm.nodes_llama_prompt_reverse import ( OUTPUT_FORMAT_HINT, _LANG_HINT, _OUTPUT_LANGS, @@ -54,10 +54,10 @@ from .nodes_llama_prompt_reverse import ( strip_reasoning, ) -from . import xb_presets +from ...libs import xb_presets # SKILL（技能文件 = LLM 的 system prompt，放 xb_skill/skills） -from .xb_skills import SKILL_NONE, skill_list, skill_sig, skill_text +from ...libs.xb_skills import SKILL_NONE, skill_list, skill_sig, skill_text # ════════════════════════════════════════════════════════════════════════════ # 本节点自带副本（精简版）：预设模式 / 预设文本 / 空latent 规格 / SKILL 三态 @@ -562,7 +562,7 @@ def task_text(label, output_lang=None): if txt: return txt try: - from .nodes_llama import preset_prompts as _builtin + from ..llm.nodes_llama import preset_prompts as _builtin return _builtin.get(label, "") or "" except Exception: return "" @@ -574,7 +574,7 @@ def register_task_preset(label, output_lang=None): if not txt: return try: - from . import nodes_llama as _nl + from ..llm import nodes_llama as _nl _nl.preset_prompts[label] = txt except Exception: pass @@ -815,7 +815,7 @@ class XB_ImagePromptPresetPro: sig = {k: kwargs.get(k) for k in keys} extra = "" try: - from .nodes_llama_prompt_reverse import read_api_settings + from ..llm.nodes_llama_prompt_reverse import read_api_settings extra = json.dumps(read_api_settings(), sort_keys=True, ensure_ascii=False) except Exception: extra = "" diff --git a/nodes_image_scale.py b/nodes/image/nodes_image_scale.py similarity index 98% rename from nodes_image_scale.py rename to nodes/image/nodes_image_scale.py index 95c9aa3..0c5c7f3 100644 --- a/nodes_image_scale.py +++ b/nodes/image/nodes_image_scale.py @@ -8,7 +8,7 @@ N进N出，共享缩放参数，每张独立处理，互不干扰。 import math import comfy.utils import nodes -from .nodes_reference_any import FlexibleOptionalInputType +from ..core.nodes_reference_any import FlexibleOptionalInputType MAX_IMAGES = 9 diff --git a/nodes_qwen_encode.py b/nodes/image/nodes_qwen_encode.py similarity index 98% rename from nodes_qwen_encode.py rename to nodes/image/nodes_qwen_encode.py index 97722e4..72bc633 100644 --- a/nodes_qwen_encode.py +++ b/nodes/image/nodes_qwen_encode.py @@ -10,7 +10,7 @@ import comfy.utils import math import torch import nodes -from .nodes_reference_any import FlexibleOptionalInputType +from ..core.nodes_reference_any import FlexibleOptionalInputType LLAMA_TEMPLATE = ( "<|im_start|>system\n" diff --git a/nodes_qwen_prompt_preset.py b/nodes/image/nodes_qwen_prompt_preset.py similarity index 99% rename from nodes_qwen_prompt_preset.py rename to nodes/image/nodes_qwen_prompt_preset.py index 587c6ef..bba96e0 100644 --- a/nodes_qwen_prompt_preset.py +++ b/nodes/image/nodes_qwen_prompt_preset.py @@ -653,11 +653,11 @@ def build_prompt(lang, preset_mode, three_view_text, body, has_image=None, io_mo # ============================================================================ # 节点 # ============================================================================ -from .xb_skills import SKILL_NONE as _SKILL_NONE -from .xb_skills import skill_list as _skill_lister -from .xb_skills import skill_path as _skill_pather -from .xb_skills import skill_sig as _skill_signer -from .xb_skills import skill_text as _read_skill_text +from ...libs.xb_skills import SKILL_NONE as _SKILL_NONE +from ...libs.xb_skills import skill_list as _skill_lister +from ...libs.xb_skills import skill_path as _skill_pather +from ...libs.xb_skills import skill_sig as _skill_signer +from ...libs.xb_skills import skill_text as _read_skill_text DEFAULT_LANG = OUTPUT_LANGS[0] # 用户要求：本节点的空latent类型默认 Qwen-image（与节点定位一致） diff --git a/nodes_segmentation.py b/nodes/image/nodes_segmentation.py similarity index 97% rename from nodes_segmentation.py rename to nodes/image/nodes_segmentation.py index 82230e0..7fa0b86 100644 --- a/nodes_segmentation.py +++ b/nodes/image/nodes_segmentation.py @@ -1,137 +1,137 @@ -""" -XB-ToolBox 人物分割节点 (ROCm/DirectML 优化) -============================================ -完全独立，零外部节点依赖。使用 onnxruntime-directml 在 AMD 显卡上 GPU 运行。 - -配套节点: - XB_HumanSegModelLoader → 加载 ONNX 模型 (文件选择器) - XB_HumanSegmentation → 人物分割 (蒙版 + 抠图) - -模型下载: https://huggingface.co/briaai/RMBG-1.4 -放入: ComfyUI/models/rembg/u2net_human_seg.onnx -""" - -import os -import numpy as np -import torch -import folder_paths - - -# 注册 rembg 模型文件夹 -if "rembg" not in folder_paths.folder_names_and_paths: - folder_paths.folder_names_and_paths["rembg"] = ( - [os.path.join(folder_paths.models_dir, "rembg")], - {".onnx"} - ) - - -def _get_ort_session(model_path: str): - """获取 ONNX Runtime 会话，优先 DirectML GPU，回退 CPU""" - import onnxruntime as ort - - providers = ort.get_available_providers() - if "DmlExecutionProvider" in providers: - print(f"[XB-Seg] ✅ DirectML GPU 加速已启用") - return ort.InferenceSession(model_path, providers=["DmlExecutionProvider"]) - elif "ROCMExecutionProvider" in providers: - print(f"[XB-Seg] ✅ ROCm GPU 加速已启用") - return ort.InferenceSession(model_path, providers=["ROCMExecutionProvider"]) - elif "CUDAExecutionProvider" in providers: - print(f"[XB-Seg] ✅ CUDA GPU 加速已启用") - return ort.InferenceSession(model_path, providers=["CUDAExecutionProvider"]) - else: - print(f"[XB-Seg] ⚠️ 未检测到 GPU 加速器，使用 CPU (速度较慢)") - return ort.InferenceSession(model_path, providers=["CPUExecutionProvider"]) - - -def _preprocess(image_bchw: torch.Tensor) -> np.ndarray: - """[B, 3, 320, 320] → numpy, ImageNet 标准化""" - mean = np.array([0.485, 0.456, 0.406], dtype=np.float32).reshape(1, 3, 1, 1) - std = np.array([0.229, 0.224, 0.225], dtype=np.float32).reshape(1, 3, 1, 1) - x = image_bchw.cpu().numpy().astype(np.float32) - return (x - mean) / std - - -def _postprocess_batch(output: np.ndarray, orig_h: int, orig_w: int, device: torch.device) -> torch.Tensor: - """ONNX 输出 [B, 1, 320, 320] → GPU 批量 resize 到 [B, H, W] mask，零 for 循环""" - mask_tensor = torch.from_numpy(output).to(device) # [B, 1, 320, 320] - mask_resized = torch.nn.functional.interpolate( - mask_tensor, size=(orig_h, orig_w), mode="bilinear", align_corners=False - ) - return torch.clamp(mask_resized.squeeze(1), 0, 1) # [B, H, W] - - -# ============================================================ -# XB_HumanSegModelLoader — 人物分割模型加载器 -# ============================================================ -class XB_HumanSegModelLoader: - """加载 u2net ONNX 模型，输出给分割节点使用""" - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model": (folder_paths.get_filename_list("rembg"),), - } - } - - RETURN_TYPES = ("XB_HUMANSEG_MODEL",) - RETURN_NAMES = ("分割模型",) - FUNCTION = "load" - CATEGORY = "XB_ToolBox/Segment" - - def load(self, model: str): - model_path = folder_paths.get_full_path_or_raise("rembg", model) - session = _get_ort_session(model_path) - print(f"[XB-Seg] 已加载: {model}") - return (session,) - - -# ============================================================ -# XB_HumanSegmentation — 人物分割 -# ============================================================ -class XB_HumanSegmentation: - """输入图像 + 分割模型，输出人物蒙版和抠图""" - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "seg_model": ("XB_HUMANSEG_MODEL",), - "image": ("IMAGE",), - } - } - - RETURN_TYPES = ("MASK", "IMAGE") - RETURN_NAMES = ("人物蒙版", "人物抠图") - FUNCTION = "segment" - CATEGORY = "XB_ToolBox/Segment" - - def segment(self, seg_model, image: torch.Tensor): - """ - seg_model: onnxruntime.InferenceSession - image: [B, H, W, C] float32 [0, 1] - """ - session = seg_model - B, orig_h, orig_w, C = image.shape - - # --- 1. 预处理: resize 到 320x320，BCHW --- - img_bchw = image.permute(0, 3, 1, 2) # [B, C, H, W] - img_bchw = torch.nn.functional.interpolate( - img_bchw, size=(320, 320), mode="bilinear", align_corners=False - ) - inp = _preprocess(img_bchw) # numpy [B, 3, 320, 320] - - # --- 2. ONNX 推理 --- - ort_inputs = {session.get_inputs()[0].name: inp} - ort_outputs = session.run(None, ort_inputs) - mask_np = ort_outputs[0] # [B, 1, 320, 320] - - # --- 3. 纯 GPU 批量后处理（一行替代 81 次 OpenCV for 循环）--- - mask_stack = _postprocess_batch(mask_np, orig_h, orig_w, image.device) # [B, H, W] - cutout_stack = image * mask_stack.unsqueeze(-1) # [B, H, W, C] 广播 - - return (mask_stack, cutout_stack) - - -__all__ = ["XB_HumanSegModelLoader", "XB_HumanSegmentation"] +""" +XB-ToolBox 人物分割节点 (ROCm/DirectML 优化) +============================================ +完全独立，零外部节点依赖。使用 onnxruntime-directml 在 AMD 显卡上 GPU 运行。 + +配套节点: + XB_HumanSegModelLoader → 加载 ONNX 模型 (文件选择器) + XB_HumanSegmentation → 人物分割 (蒙版 + 抠图) + +模型下载: https://huggingface.co/briaai/RMBG-1.4 +放入: ComfyUI/models/rembg/u2net_human_seg.onnx +""" + +import os +import numpy as np +import torch +import folder_paths + + +# 注册 rembg 模型文件夹 +if "rembg" not in folder_paths.folder_names_and_paths: + folder_paths.folder_names_and_paths["rembg"] = ( + [os.path.join(folder_paths.models_dir, "rembg")], + {".onnx"} + ) + + +def _get_ort_session(model_path: str): + """获取 ONNX Runtime 会话，优先 DirectML GPU，回退 CPU""" + import onnxruntime as ort + + providers = ort.get_available_providers() + if "DmlExecutionProvider" in providers: + print(f"[XB-Seg] ✅ DirectML GPU 加速已启用") + return ort.InferenceSession(model_path, providers=["DmlExecutionProvider"]) + elif "ROCMExecutionProvider" in providers: + print(f"[XB-Seg] ✅ ROCm GPU 加速已启用") + return ort.InferenceSession(model_path, providers=["ROCMExecutionProvider"]) + elif "CUDAExecutionProvider" in providers: + print(f"[XB-Seg] ✅ CUDA GPU 加速已启用") + return ort.InferenceSession(model_path, providers=["CUDAExecutionProvider"]) + else: + print(f"[XB-Seg] ⚠️ 未检测到 GPU 加速器，使用 CPU (速度较慢)") + return ort.InferenceSession(model_path, providers=["CPUExecutionProvider"]) + + +def _preprocess(image_bchw: torch.Tensor) -> np.ndarray: + """[B, 3, 320, 320] → numpy, ImageNet 标准化""" + mean = np.array([0.485, 0.456, 0.406], dtype=np.float32).reshape(1, 3, 1, 1) + std = np.array([0.229, 0.224, 0.225], dtype=np.float32).reshape(1, 3, 1, 1) + x = image_bchw.cpu().numpy().astype(np.float32) + return (x - mean) / std + + +def _postprocess_batch(output: np.ndarray, orig_h: int, orig_w: int, device: torch.device) -> torch.Tensor: + """ONNX 输出 [B, 1, 320, 320] → GPU 批量 resize 到 [B, H, W] mask，零 for 循环""" + mask_tensor = torch.from_numpy(output).to(device) # [B, 1, 320, 320] + mask_resized = torch.nn.functional.interpolate( + mask_tensor, size=(orig_h, orig_w), mode="bilinear", align_corners=False + ) + return torch.clamp(mask_resized.squeeze(1), 0, 1) # [B, H, W] + + +# ============================================================ +# XB_HumanSegModelLoader — 人物分割模型加载器 +# ============================================================ +class XB_HumanSegModelLoader: + """加载 u2net ONNX 模型，输出给分割节点使用""" + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": (folder_paths.get_filename_list("rembg"),), + } + } + + RETURN_TYPES = ("XB_HUMANSEG_MODEL",) + RETURN_NAMES = ("分割模型",) + FUNCTION = "load" + CATEGORY = "XB_ToolBox/Segment" + + def load(self, model: str): + model_path = folder_paths.get_full_path_or_raise("rembg", model) + session = _get_ort_session(model_path) + print(f"[XB-Seg] 已加载: {model}") + return (session,) + + +# ============================================================ +# XB_HumanSegmentation — 人物分割 +# ============================================================ +class XB_HumanSegmentation: + """输入图像 + 分割模型，输出人物蒙版和抠图""" + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "seg_model": ("XB_HUMANSEG_MODEL",), + "image": ("IMAGE",), + } + } + + RETURN_TYPES = ("MASK", "IMAGE") + RETURN_NAMES = ("人物蒙版", "人物抠图") + FUNCTION = "segment" + CATEGORY = "XB_ToolBox/Segment" + + def segment(self, seg_model, image: torch.Tensor): + """ + seg_model: onnxruntime.InferenceSession + image: [B, H, W, C] float32 [0, 1] + """ + session = seg_model + B, orig_h, orig_w, C = image.shape + + # --- 1. 预处理: resize 到 320x320，BCHW --- + img_bchw = image.permute(0, 3, 1, 2) # [B, C, H, W] + img_bchw = torch.nn.functional.interpolate( + img_bchw, size=(320, 320), mode="bilinear", align_corners=False + ) + inp = _preprocess(img_bchw) # numpy [B, 3, 320, 320] + + # --- 2. ONNX 推理 --- + ort_inputs = {session.get_inputs()[0].name: inp} + ort_outputs = session.run(None, ort_inputs) + mask_np = ort_outputs[0] # [B, 1, 320, 320] + + # --- 3. 纯 GPU 批量后处理（一行替代 81 次 OpenCV for 循环）--- + mask_stack = _postprocess_batch(mask_np, orig_h, orig_w, image.device) # [B, H, W] + cutout_stack = image * mask_stack.unsqueeze(-1) # [B, H, W, C] 广播 + + return (mask_stack, cutout_stack) + + +__all__ = ["XB_HumanSegModelLoader", "XB_HumanSegmentation"] diff --git a/nodes_shenbi.py b/nodes/image/nodes_shenbi.py similarity index 100% rename from nodes_shenbi.py rename to nodes/image/nodes_shenbi.py diff --git a/nodes/llm/__init__.py b/nodes/llm/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/nodes_llama.py b/nodes/llm/nodes_llama.py similarity index 99% rename from nodes_llama.py rename to nodes/llm/nodes_llama.py index a38c558..7b07a87 100644 --- a/nodes_llama.py +++ b/nodes/llm/nodes_llama.py @@ -32,10 +32,10 @@ import torch import numpy as np from PIL import Image, ImageDraw from scipy.ndimage import gaussian_filter -from .support_llama.cqdm import cqdm -from .support_llama.gguf_layers import get_layer_count -from .support_llama.prompt_enhancer_preset import * -from .support_llama.presets.storyboard import ( +from ...support_llama.cqdm import cqdm +from ...support_llama.gguf_layers import get_layer_count +from ...support_llama.prompt_enhancer_preset import * +from ...support_llama.presets.storyboard import ( STORYBOARD_SKELETON, MODEL_RULES, STYLE_RULES, CAMERA_LOGIC, LANGUAGE_RULES, build_storyboard_prompt ) diff --git a/nodes_llama_prompt_reverse.py b/nodes/llm/nodes_llama_prompt_reverse.py similarity index 99% rename from nodes_llama_prompt_reverse.py rename to nodes/llm/nodes_llama_prompt_reverse.py index 953840b..67d5121 100644 --- a/nodes_llama_prompt_reverse.py +++ b/nodes/llm/nodes_llama_prompt_reverse.py @@ -55,7 +55,7 @@ def _api_settings_file(): try: base = folder_paths.get_user_directory() except Exception: - base = os.path.dirname(os.path.abspath(__file__)) + base = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) return os.path.join(base, "xb_toolbox_llm_api.json") diff --git a/nodes_model_loader_gguf.py b/nodes/llm/nodes_model_loader_gguf.py similarity index 99% rename from nodes_model_loader_gguf.py rename to nodes/llm/nodes_model_loader_gguf.py index cfca457..04e12c6 100644 --- a/nodes_model_loader_gguf.py +++ b/nodes/llm/nodes_model_loader_gguf.py @@ -108,7 +108,7 @@ def _apply_sage_attention(model, preset: str): if preset == "关闭" or model is None: return model try: - from .nodes_sageatt import XB_SageAttentionAccelerator + from ..core.nodes_sageatt import XB_SageAttentionAccelerator return XB_SageAttentionAccelerator().patch(model, preset)[0] except Exception: return model @@ -118,7 +118,7 @@ def _apply_block_swap(model, blocks_to_swap: int): if blocks_to_swap == 0 or model is None: return model try: - from .nodes_blockswap import XB_UNetBlockSwap + from ..core.nodes_blockswap import XB_UNetBlockSwap return XB_UNetBlockSwap().set_callback(model, blocks_to_swap)[0] except Exception: return model diff --git a/nodes_model_loader_v1.py b/nodes/llm/nodes_model_loader_v1.py similarity index 100% rename from nodes_model_loader_v1.py rename to nodes/llm/nodes_model_loader_v1.py diff --git a/nodes/video/__init__.py b/nodes/video/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/nodes_digital_human.py b/nodes/video/nodes_digital_human.py similarity index 99% rename from nodes_digital_human.py rename to nodes/video/nodes_digital_human.py index e1ebb41..4aab8ea 100644 --- a/nodes_digital_human.py +++ b/nodes/video/nodes_digital_human.py @@ -10,7 +10,7 @@ import os import folder_paths import torch -from .nodes_audio_slicer import _load_audio_file, _make_audio, _snap_4n1 +from ..audio.nodes_audio_slicer import _load_audio_file, _make_audio, _snap_4n1 # ============================================================ diff --git a/nodes_hailuo_video.py b/nodes/video/nodes_hailuo_video.py similarity index 100% rename from nodes_hailuo_video.py rename to nodes/video/nodes_hailuo_video.py diff --git a/nodes_load_video.py b/nodes/video/nodes_load_video.py similarity index 100% rename from nodes_load_video.py rename to nodes/video/nodes_load_video.py diff --git a/nodes_ltx_streamer.py b/nodes/video/nodes_ltx_streamer.py similarity index 100% rename from nodes_ltx_streamer.py rename to nodes/video/nodes_ltx_streamer.py diff --git a/nodes_pipeline.py b/nodes/video/nodes_pipeline.py similarity index 97% rename from nodes_pipeline.py rename to nodes/video/nodes_pipeline.py index fb4770f..136c24e 100644 --- a/nodes_pipeline.py +++ b/nodes/video/nodes_pipeline.py @@ -1,2506 +1,2506 @@ -import os -import torch -import numpy as np -import gc -from PIL import Image, ImageOps -import nodes -import comfy.samplers -import comfy.model_management as mm -import folder_paths -try: - import torchaudio -except ImportError: - torchaudio = None - print("⚠️ [XB-BOX] torchaudio 未安装！音频重采样功能将不可用。请运行: pip install torchaudio") -from .nodes_wan_vae import XB_WanFirstLastFrameToVideo, XB_WanImageToVideo, XB_WanInfiniteTalkToVideo_Single - - -# VRAM 保护阈值：累积视频超过总显存此比例时自动迁移至 CPU（兼容 6GB~48GB 全系列显卡） -_VRAM_VIDEO_SAFETY_RATIO = 0.30 - - -def _get_vram_info(): - """获取当前 CUDA 设备显存信息 (total, free, used)""" - if not torch.cuda.is_available(): - return 0, 0, 0 - d = torch.cuda.current_device() - total = torch.cuda.get_device_properties(d).total_memory - reserved = torch.cuda.memory_reserved(d) - allocated = torch.cuda.memory_allocated(d) - free = total - reserved - return total, free, allocated - - -def _refresh_models(force=False): - """VRAM 感知缓存刷新：仅在显存紧张时清理，避免频繁清空分配池导致碎片化""" - if not force: - _, free, _ = _get_vram_info() - if torch.cuda.is_available(): - total = torch.cuda.get_device_properties(torch.cuda.current_device()).total_memory - if free > total * 0.12: - return - if torch.cuda.is_available(): - torch.cuda.synchronize() - torch.cuda.empty_cache() - elif torch.backends.mps.is_available(): - torch.mps.synchronize() - torch.mps.empty_cache() - gc.collect() - - -def _safe_video_accumulate(prev_video, new_segment, concat_dim, label="", concat_mode="自动"): - """ - VRAM 感知视频累积拼接，兼容从 6GB 到 48GB 的所有配置。 - concat_mode: "自动"(默认) | "强制GPU" | "强制CPU" - """ - if prev_video is None: - return new_segment - - target_device = new_segment.device - - # 强制CPU - if concat_mode == "强制CPU": - new_segment = new_segment.cpu() - if prev_video.device.type != 'cpu': - prev_video = prev_video.cpu() - result = torch.cat([prev_video, new_segment], dim=concat_dim) - tag = f"[{label}] " if label else "" - print(f"💾 [XB-BOX] {tag}强制CPU拼接") - return result - - # 强制GPU - if concat_mode == "强制GPU": - if prev_video.device != target_device: - prev_video = prev_video.to(target_device, non_blocking=True) - result = torch.cat([prev_video, new_segment], dim=concat_dim) - tag = f"[{label}] " if label else "" - print(f"🚀 [XB-BOX] {tag}强制GPU拼接: {result.numel()*result.element_size()/1024**3:.2f}GB") - return result - - # 自动模式 - combined_bytes = (prev_video.numel() + new_segment.numel()) * new_segment.element_size() - if target_device.type == 'cuda': - total_mem = torch.cuda.get_device_properties(target_device).total_memory - ratio = combined_bytes / total_mem - else: - ratio = 0.0 - - if prev_video.device.type == 'cpu' or ratio > _VRAM_VIDEO_SAFETY_RATIO: - new_segment = new_segment.cpu() - if prev_video.device.type != 'cpu': - prev_video = prev_video.cpu() - result = torch.cat([prev_video, new_segment], dim=concat_dim) - tag = f"[{label}] " if label else "" - print(f"💾 [XB-BOX] {tag}VRAM保护: 累积视频 {combined_bytes/1024**3:.2f}GB ({ratio*100:.0f}% 总显存) → 已在 CPU 完成无损拼接") - return result - - if prev_video.device != target_device: - prev_video = prev_video.to(target_device, non_blocking=True) - return torch.cat([prev_video, new_segment], dim=concat_dim) - - -def _match_color_to_ref(target_img, ref_img): - """色彩重映射引擎：用均值和标准差将 target 光影强制对齐到 ref。 - - 3D VAE 的边界帧因缺失未来帧 padding 会变暗——此函数用数学手段 - 把变暗的尾帧拉回原始首帧的光影分布，斩断接力中的曝光衰减死循环。 - - target_img: [1, H, W, 3] (变暗的尾帧) - ref_img: [1, H', W', 3] (总线里的全局参考首帧) - """ - t = target_img.movedim(-1, 1) # [1, 3, H, W] - r = ref_img.movedim(-1, 1) # [1, 3, H', W'] - - t_mean = t.mean(dim=(2, 3), keepdim=True) - t_std = t.std(dim=(2, 3), keepdim=True) + 1e-6 - r_mean = r.mean(dim=(2, 3), keepdim=True) - r_std = r.std(dim=(2, 3), keepdim=True) + 1e-6 - - matched = (t - t_mean) / t_std * r_std + r_mean - return torch.clamp(matched, 0.0, 1.0).movedim(1, -1) - - -def _apply_progressive_color_correction(decoded_image, ref_image, is_4d): - """全序列渐变色彩补偿：平方权重 + 50% 混合降敏，避免尾端色彩过度补偿伪影。 - - 用 start_image 做色彩锚点。平方权重 (x²) 比三次幂 (x³) 过渡更平缓， - 在 33 帧场景下后 15% 帧的修正强度从 73% 降至 60%。 - 50% blending (原帧×0.5 + 匹配帧×0.5) 进一步抑制 color transfer 的内容差异伪影。 - """ - vid = decoded_image if is_4d else decoded_image[0] - T = vid.shape[0] - if T <= 1: - return decoded_image - - raw_last = vid[-1:] # [1, H, W, C] - full_match = _match_color_to_ref(raw_last, ref_image) - - # 🌟 50% 混合降敏：一半原帧 + 一半色彩匹配，抑制内容差异导致的过度补偿 - blended_last = raw_last * 0.5 + full_match * 0.5 - - drift_diff = blended_last - raw_last - linear = torch.linspace(0.0, 1.0, T, device=vid.device, dtype=vid.dtype) - weights = (linear ** 2).view(T, 1, 1, 1) # 平方权重，更平缓 - - corrected_vid = vid + drift_diff * weights - corrected_vid = torch.clamp(corrected_vid, 0.0, 1.0) - - return corrected_vid if is_4d else corrected_vid.unsqueeze(0) - - -# ============================================================ -# XB_Wan_ParamBus — Wan 视频参数总线 -# ============================================================ -class XB_Wan_ParamBus: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model_high": ("MODEL",), - "model_low": ("MODEL",), - "clip": ("CLIP",), - "vae": ("VAE",), - "negative_prompt": ("STRING", {"multiline": True, "default": "vivid colors, overexposed, static, blurry details..."}), - "width": ("INT", {"forceInput": True}), - "height": ("INT", {"forceInput": True}), - "length": ("INT", {"forceInput": True}), - "fps": ("FLOAT", {"forceInput": True}), - # --- VAE 时空分块与重叠参数 --- - "vae_encode_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - "vae_decode_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), - "temporal_chunk_size": ("INT", {"default": 64, "min": 0, "max": 8192, "step": 4}), - "temporal_overlap": ("INT", {"default": 8, "min": 0, "max": 8192, "step": 4}), - "steps": ("INT", {"default": 4, "min": 1, "max": 100}), - "high_noise_steps": ("INT", {"default": 2, "min": 1, "max": 50}), - "cfg": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.1}), - "sampler_name": (comfy.samplers.KSAMPLER_NAMES, ), - "scheduler": (comfy.samplers.SCHEDULER_NAMES, ), - "seed": ("INT", {"default": 123456789}), - "cut_first_frame": ("BOOLEAN", {"default": True, "label_on": "Yes (Relay Deduplication)", "label_off": "No (Keep First Frame)"}), - "trim_head_frames": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), - }, - "optional": { - "clip_vision": ("CLIP_VISION",), - "global_start_image": ("IMAGE",), - "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), - "scale_method": (["lanczos", "bilinear", "bicubic", "nearest-exact", "area"], {"default": "lanczos"}), - "crop_mode": (["center", "disabled"], {"default": "center"}), - } - } - - RETURN_TYPES = ("WAN_BUS",) - RETURN_NAMES = ("📦 WAN_BUS (Dynamic Bus)",) - FUNCTION = "pack_bus" - CATEGORY = "XB_ToolBox/Pipeline" - - def pack_bus(self, **kwargs): - import comfy.utils - cv = kwargs.get("clip_vision") - ref = kwargs.get("global_start_image") # 首尾帧/图生视频用 global_start_image 做视觉参考 - if cv is not None and ref is not None: - w, h = kwargs["width"], kwargs["height"] - method = kwargs.get("scale_method", "lanczos") - crop = kwargs.get("crop_mode", "center") - scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) - kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") - print(f"👁️ [XB-BOX] CLIP视觉编码 ({ref.shape[1]}×{ref.shape[2]} → {w}×{h}, {method}/{crop})") - else: - kwargs["clip_vision_output"] = None - return (kwargs,) - - -# ============================================================ -# XB_Wan_RelayNode — Wan 首尾帧接力点 -# ============================================================ -class XB_Wan_RelayNode: - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] - files = sorted(files) - - if not files: - files = ["[Folder empty_Please connect or upload]"] - - return { - "required": { - "wan_bus": ("WAN_BUS",), - "start_image": ("IMAGE",), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), - "end_image_file": (files, {"image_upload": True}) - }, - "optional": { - "prev_video": ("IMAGE",), - "opt_end_image": ("IMAGE",) - } - } - - @classmethod - def VALIDATE_INPUTS(cls, **kwargs): - return True - - RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") - RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_bus, start_image, positive_prompt, end_image_file, opt_end_image=None, prev_video=None): - print("\n🏃 [XB-BOX] Executing First/Last frame constraint relay task...") - b = wan_bus - seed = b["seed"] - cut_first_frame = b.get("cut_first_frame", True) - trim_head_frames = b.get("trim_head_frames", 1) - - encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) - decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) - spat_overlap = b.get("spatial_overlap", 32) - temp_chunk = b.get("temporal_chunk_size", 64) - temp_overlap = b.get("temporal_overlap", 8) - - if opt_end_image is not None: - end_image = opt_end_image - print(f"🖼️ [XB-BOX] Endpoint connection detected, prioritizing opt_end_image payload.") - else: - if end_image_file == "[Folder empty_Please connect or upload]": - print("⏭️ [XB-BOX] No end frame specified for this segment, skipping relay iteration.") - return (wan_bus, start_image, prev_video) - - image_path = folder_paths.get_annotated_filepath(end_image_file) - if not os.path.exists(image_path): - print(f"⏭️ [XB-BOX] End frame file not found: {os.path.basename(image_path)}, skipping relay iteration.") - return (wan_bus, start_image, prev_video) - - i = Image.open(image_path) - i = ImageOps.exif_transpose(i) - image_data = i.convert("RGB") - image_data = np.array(image_data).astype(np.float32) / 255.0 - end_image = torch.from_numpy(image_data)[None,] - - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - cv_output = b.get("clip_vision_output") - pos, neg, latent = XB_WanFirstLastFrameToVideo().process( - positive=pos_cond, negative=neg_cond, vae=b["vae"], - clip_vision_start_image=cv_output, clip_vision_end_image=None, - start_image=start_image, end_image=end_image, - width=b["width"], height=b["height"], length=b["length"], - batch_size=1, vae_tile_size=encode_tile, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - # 🛡️ 护城河：步数边界校验，防止 ODE 时间步崩塌 - total_steps = b["steps"] - high_steps = b["high_noise_steps"] - if high_steps >= total_steps: - high_steps = total_steps - 1 - print(f"⚠️ [XB-BOX] 拦截到致命参数：强制修正 high_noise_steps 为 {high_steps}") - - print(f"🔥 [XB-BOX] Samping Layer 1 (0 -> {high_steps} steps)...") - latent_high, = nodes.KSamplerAdvanced().sample( - model=b["model_high"], add_noise="enable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, - start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" - ) - - print(f"❄️ [XB-BOX] Samping Layer 2 ({high_steps} -> {total_steps})...") - latent_low, = nodes.KSamplerAdvanced().sample( - model=b["model_low"], add_noise="disable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent_high, - start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" - ) - - decoded_image, = nodes.VAEDecode().decode( - samples=latent_low, - vae=b["vae"], - ) - - is_4d = len(decoded_image.shape) == 4 - frame_dim = 0 if is_4d else 1 - total_frames = decoded_image.shape[frame_dim] - - if cut_first_frame and total_frames > 1: - safe_trim = min(trim_head_frames, total_frames - 1) - if safe_trim > 0: - if is_4d: - decoded_image = decoded_image[safe_trim:, :, :, :] - else: - decoded_image = decoded_image[:, safe_trim:, :, :, :] - - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - - # 👑 全序列渐变补偿：start_image 做锚点（接力中=上一段修正尾帧，同色彩空间） - decoded_image = _apply_progressive_color_correction(decoded_image, start_image, is_4d) - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - print("✨ [XB-BOX] 触发抗衰减机制：已应用全序列 3D VAE 曝光漂移渐变补偿！") - - concat_dim = 0 if is_4d else 1 - - final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="Relay") - - _refresh_models() - return (wan_bus, last_frame, final_video) - - -# ============================================================ -# XB_Wan_InfiniteRelayNode — Wan 无限长接力点 -# ============================================================ -class XB_Wan_InfiniteRelayNode: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "wan_bus": ("WAN_BUS",), - "start_image": ("IMAGE",), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), - }, - "optional": { - "prev_video": ("IMAGE",) - } - } - - @classmethod - def VALIDATE_INPUTS(cls, **kwargs): - return True - - RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") - RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_bus, start_image, positive_prompt, prev_video=None): - print("\n🏃‍♀️ [XB-BOX] Executing Image-to-Video Infinite pipeline generation task...") - b = wan_bus - seed = b["seed"] - cut_first_frame = b.get("cut_first_frame", True) - trim_head_frames = b.get("trim_head_frames", 1) - - encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) - decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) - spat_overlap = b.get("spatial_overlap", 32) - temp_chunk = b.get("temporal_chunk_size", 64) - temp_overlap = b.get("temporal_overlap", 8) - - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - cv_output = b.get("clip_vision_output") - pos, neg, latent = XB_WanImageToVideo().process( - positive=pos_cond, negative=neg_cond, vae=b["vae"], - clip_vision_output=cv_output, start_image=start_image, - width=b["width"], height=b["height"], length=b["length"], - batch_size=1, vae_tile_size=encode_tile, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - # 🛡️ 护城河：步数边界校验，防止 ODE 时间步崩塌 - total_steps = b["steps"] - high_steps = b["high_noise_steps"] - if high_steps >= total_steps: - high_steps = total_steps - 1 - print(f"⚠️ [XB-BOX] 拦截到致命参数：强制修正 high_noise_steps 为 {high_steps}") - - print(f"🔥 [XB-BOX] Samping Layer 1 (0 -> {high_steps} steps)...") - latent_high, = nodes.KSamplerAdvanced().sample( - model=b["model_high"], add_noise="enable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, - start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" - ) - - print(f"❄️ [XB-BOX] Samping Layer 2 ({high_steps} -> {total_steps})...") - latent_low, = nodes.KSamplerAdvanced().sample( - model=b["model_low"], add_noise="disable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent_high, - start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" - ) - - decoded_image, = nodes.VAEDecode().decode( - samples=latent_low, - vae=b["vae"], - ) - - is_4d = len(decoded_image.shape) == 4 - frame_dim = 0 if is_4d else 1 - total_frames = decoded_image.shape[frame_dim] - - if cut_first_frame and total_frames > 1: - safe_trim = min(trim_head_frames, total_frames - 1) - if safe_trim > 0: - if is_4d: - decoded_image = decoded_image[safe_trim:, :, :, :] - else: - decoded_image = decoded_image[:, safe_trim:, :, :, :] - - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - - # 👑 全序列渐变补偿：start_image 做锚点（接力中=上一段修正尾帧，同色彩空间） - decoded_image = _apply_progressive_color_correction(decoded_image, start_image, is_4d) - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - print("✨ [XB-BOX] 触发抗衰减机制：已应用全序列 3D VAE 曝光漂移渐变补偿！") - - concat_dim = 0 if is_4d else 1 - - final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="InfiniteRelay") - - _refresh_models() - return (wan_bus, last_frame, final_video) - - -# ============================================================ -# XB_Video_Merger — 视频拼接器 -# ============================================================ -class XB_Video_Merger: - @classmethod - def INPUT_TYPES(cls): - inputs = {"required": {}, "optional": {}} - for i in range(1, 11): - inputs["optional"][f"video_{i}"] = ("IMAGE",) - return inputs - - RETURN_TYPES = ("IMAGE",) - RETURN_NAMES = ("🎞️ Full Long Video",) - FUNCTION = "merge_videos" - CATEGORY = "XB_ToolBox/Pipeline" - - def merge_videos(self, **kwargs): - videos = [kwargs.get(f"video_{i}") for i in range(1, 11) if kwargs.get(f"video_{i}") is not None] - if not videos: - raise ValueError("🚨 [XB-BOX] Merge failed: At least one video must be connected!") - - concat_dim = 0 if len(videos[0].shape) == 4 else 1 - ref_shape = list(videos[0].shape) - for i, vid in enumerate(videos): - for d in range(len(ref_shape)): - if d != concat_dim and vid.shape[d] != ref_shape[d]: - raise ValueError( - f"🚨 [XB-BOX] Merge failed: Video segment {i+1} shape {list(vid.shape)} " - f"does not match first segment shape {ref_shape} on dimension {d} " - f"(expected {ref_shape[d]}, got {vid.shape[d]})" - ) - # 将全部视频段卸载到 CPU 再拼接，防止多段同时驻留 VRAM 导致 OOM - videos_cpu = [v.cpu() for v in videos] - merged = torch.cat(videos_cpu, dim=concat_dim) - print(f"💾 [XB-BOX] Merger: {len(videos)} 段视频已在 CPU 完成拼接 (shape={list(merged.shape)})") - return (merged,) - -# ============================================================ -# XB_StoryboardSlicer — 分镜切片器 -# ============================================================ -class XB_StoryboardSlicer: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "image": ("IMAGE",), - "mode": (["3-Grid (1x3 Horizontal)", "3-Grid (3x1 Vertical)", "4-Grid (2x2)", "6-Grid (2x3 Horizontal)", "6-Grid (3x2 Vertical)", "9-Grid (3x3)"], {"default": "4-Grid (2x2)"}), - "crop_margin": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 0.45, "step": 0.01}), - } - } - RETURN_TYPES = ("IMAGE",) * 9 - RETURN_NAMES = ("Img1", "Img2", "Img3", "Img4", "Img5", "Img6", "Img7", "Img8", "Img9") - FUNCTION = "slice_image" - CATEGORY = "XB_ToolBox/Pipeline" - - def slice_image(self, image, mode, crop_margin): - if "1x3" in mode: rows, cols = 1, 3 - elif "3x1" in mode: rows, cols = 3, 1 - elif "2x2" in mode: rows, cols = 2, 2 - elif "2x3" in mode: rows, cols = 2, 3 - elif "3x2" in mode: rows, cols = 3, 2 - elif "3x3" in mode: rows, cols = 3, 3 - else: rows, cols = 2, 2 - - batch_size, h, w, channels = image.shape - slice_h, slice_w = h // rows, w // cols - crop_y, crop_x = int(slice_h * crop_margin), int(slice_w * crop_margin) - - sliced_images = [] - for r in range(rows): - for c in range(cols): - sy, ey = r * slice_h + crop_y, (r + 1) * slice_h - crop_y - sx, ex = c * slice_w + crop_x, (c + 1) * slice_w - crop_x - if sy >= ey: ey = sy + 1 - if sx >= ex: ex = sx + 1 - sliced_images.append(image[:, sy:ey, sx:ex, :]) - - placeholder_h = max(1, slice_h - crop_y * 2) - placeholder_w = max(1, slice_w - crop_x * 2) - while len(sliced_images) < 9: - sliced_images.append(torch.zeros((1, placeholder_h, placeholder_w, 3), dtype=torch.float32, device=image.device)) - return tuple(sliced_images) - - -# ============================================================ -# XB_WanAnimate_ParamBus — Wan Animate 动作迁移参数总线 -# ============================================================ -class XB_WanAnimate_ParamBus: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model": ("MODEL",), - "clip": ("CLIP",), - "vae": ("VAE",), - "negative_prompt": ("STRING", {"multiline": True, "default": "vivid colors, overexposed, static, blurry details..."}), - "width": ("INT", {"default": 480, "min": 260, "step": 16}), - "height": ("INT", {"default": 832, "step": 16}), - "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "0=自动从源视频取长度；>0=强制截断到此帧数"}), - "fps": ("FLOAT", {"default": 16.0}), - # --- VAE 时空分块与重叠参数 --- - "vae_encode_tile_size": ("INT", {"default": 320, "step": 32}), - "vae_decode_tile_size": ("INT", {"default": 320, "step": 32}), - "spatial_overlap": ("INT", {"default": 32, "step": 32}), - "temporal_chunk_size": ("INT", {"default": 64, "step": 4}), - "temporal_overlap": ("INT", {"default": 8, "step": 4}), - # --- 采样与控制 --- - "steps": ("INT", {"default": 20, "min": 1, "max": 100}), - "cfg": ("FLOAT", {"default": 5.0, "step": 0.1}), - "sampler_name": (comfy.samplers.KSAMPLER_NAMES, ), - "scheduler": (comfy.samplers.SCHEDULER_NAMES, ), - "seed": ("INT", {"default": 123456789}), - "continue_motion_max_frames": ("INT", {"default": 5, "min": 0, "max": 16}), - # --- 显存清理 --- - "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), - # --- CLIP视觉 --- - "clip_vision": ("CLIP_VISION",), - }, - "optional": { - "global_ref_image": ("IMAGE",), - "pose_video": ("IMAGE",), - "face_video": ("IMAGE",), - "background_video": ("IMAGE",), - "character_mask": ("MASK",), - "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), - "scale_method": (["lanczos", "bilinear", "bicubic", "nearest-exact", "area"], {"default": "lanczos"}), - "crop_mode": (["center", "disabled"], {"default": "center"}), - } - } - - RETURN_TYPES = ("WAN_ANIMATE_BUS",) - RETURN_NAMES = ("📦 WAN_ANIMATE_BUS",) - FUNCTION = "pack_bus" - CATEGORY = "XB_ToolBox/Pipeline" - - def pack_bus(self, **kwargs): - kwargs["current_offset"] = 0 - cv = kwargs.get("clip_vision") - ref = kwargs.get("global_ref_image") - if cv is not None and ref is not None: - w, h = kwargs.get("width", 480), kwargs.get("height", 832) - m = kwargs.get("scale_method", "lanczos") - c = kwargs.get("crop_mode", "center") - scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, m, c).movedim(1, -1) - kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") - print(f"👁️ [XB-BOX] Animate CLIP视觉编码 ({ref.shape[2]}×{ref.shape[1]} → {w}×{h}, {m}/{c})") - return (kwargs,) - - -# ============================================================ -# XB_WanAnimate_RelayNode — Wan Animate 无限接力点 -# ============================================================ -class XB_WanAnimate_RelayNode: - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] - files = sorted(files) - - if not files: - files = ["[Folder empty_Please connect or upload]"] - - return { - "required": { - "wan_animate_bus": ("WAN_ANIMATE_BUS",), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), - "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), - "ref_image_file": (files, {"image_upload": True}), - }, - "optional": { - "prev_video": ("IMAGE",), - } - } - - @classmethod - def VALIDATE_INPUTS(cls, **kwargs): - return True - - RETURN_TYPES = ("WAN_ANIMATE_BUS", "IMAGE") - RETURN_NAMES = ("📦 WAN_ANIMATE_BUS (传给下段)", "🎞️ 累加视频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_animate_bus, segment_length, positive_prompt, use_local_ref_image, ref_image_file, prev_video=None, opt_ref_image=None): - b = wan_animate_bus.copy() - current_offset = b.get("current_offset", 0) - seed = b.get("seed", 123456789) - - source_video = b.get("pose_video") if b.get("pose_video") is not None else b.get("face_video") - total_source_frames = float('inf') - - # 优先使用总线 total_frames，其次用源视频长度 - bus_total = b.get("total_frames", 0) - if bus_total > 0: - total_source_frames = bus_total - elif source_video is not None: - total_source_frames = source_video.shape[0] if len(source_video.shape) == 4 else source_video.shape[1] - - remaining_frames = max(0, total_source_frames - current_offset) - cont_max = b.get("continue_motion_max_frames", 5) - - if remaining_frames <= 0: - print(f"\n⏭️ [XB-BOX] 动作视频已耗尽 (已跑到 {current_offset} 帧 / 总 {total_source_frames} 帧)。当前接力点直接跳过！") - fallback_video = prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3)) - return (b, fallback_video) - - if remaining_frames < segment_length: - # 最后一段：trim 会吃掉 continue_motion 帧，需补回；再对齐 4N+1 - if current_offset == 0: - need_raw = remaining_frames # 第一段无 trim - else: - need_raw = remaining_frames + cont_max - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - print(f"⏭️ [XB-BOX] 剩余 {remaining_frames} 帧不足以生成最小段，跳过") - fallback_video = prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3)) - return (b, fallback_video) - print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining_frames} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - print(f"\n🏃‍♀️ [XB-BOX] Executing Wan Animate Relay task... Target segment length: {actual_length}") - - is_local = (use_local_ref_image == "独立参考图" or - use_local_ref_image is True or - use_local_ref_image == 1) - if is_local: - if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": - print("⏭️ [XB-BOX] 已开启独立参考图但未选择图片文件，跳过当前 Animate 接力点。") - return (b, prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3))) - image_path = folder_paths.get_annotated_filepath(ref_image_file) - if not os.path.exists(image_path): - print(f"⏭️ [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}，跳过当前 Animate 接力点。") - return (b, prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3))) - i = Image.open(image_path) - i = ImageOps.exif_transpose(i) - image_data = i.convert("RGB") - image_data = np.array(image_data).astype(np.float32) / 255.0 - ref_image = torch.from_numpy(image_data)[None,] - cv_model = b.get("clip_vision") - if cv_model is None: - raise ValueError("🚨 [XB-BOX] 使用独立参考图需要总线连接 clip_vision 模型！") - # CLIP 编码用总线缩放参数，不做预缩放（XB_WanAnimateToVideo 内部自己处理 VAE 编码） - w, h = b.get("width", 480), b.get("height", 832) - sm = b.get("scale_method", "lanczos") - cm = b.get("crop_mode", "center") - clip_img = comfy.utils.common_upscale(ref_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) - clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") - print("🖼️ [XB-BOX] 使用独立参考图（含独立CLIP视觉编码）。") - else: - ref_image = b.get("global_ref_image") - if ref_image is None: - raise ValueError("🚨 [XB-BOX] 缺少参考图！请开启独立参考图选择图片，或在总线上连接 global_ref_image。") - clip_vision_output = b.get("clip_vision_output") - print("🌐 [XB-BOX] 使用总线全局参考图。") - - prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) - - continue_motion = None - cont_max_frames = b.get("continue_motion_max_frames", 5) - if is_local: - cont_max_frames = 0 # 独立参考图时关闭衔接帧，避免CLIP视觉特征冲突 - if prev_video is not None and cont_max_frames > 0: - is_4d = len(prev_video.shape) == 4 - if is_4d: - continue_motion = prev_video[-cont_max_frames:, :, :, :] - else: - continue_motion = prev_video[:, -cont_max_frames:, :, :, :] - - try: - from .nodes_wan import XB_WanAnimateToVideo - except ImportError: - raise ImportError("🚨 [XB-BOX] 找不到 XB_WanAnimateToVideo 底座节点，请检查 nodes_wan.py 文件目录！") - - animate_core = XB_WanAnimateToVideo() - func_name = getattr(XB_WanAnimateToVideo, "FUNCTION", "process") - func = getattr(animate_core, func_name) - - kwargs = { - "positive": pos_cond, - "negative": neg_cond, - "vae": b["vae"], - "clip_vision_output": clip_vision_output, - "reference_image": ref_image, - "face_video": b.get("face_video"), - "pose_video": b.get("pose_video"), - "background_video": b.get("background_video"), - "character_mask": b.get("character_mask"), - "continue_motion": continue_motion, - "width": b["width"], - "height": b["height"], - "length": actual_length, - "batch_size": 1, - "continue_motion_max_frames": cont_max_frames, - "video_frame_offset": current_offset, - "vae_tile_size": b.get("vae_encode_tile_size", 320) - } - - pos, neg, latent, trim_latent, trim_image, new_offset = func(**kwargs) - - print(f"🔥 [XB-BOX] Animate Single-Pass Sampling ({b['steps']} steps)...") - latent_sampled, = nodes.KSampler().sample( - model=b["model"], - seed=seed, - steps=b["steps"], - cfg=b["cfg"], - sampler_name=b["sampler_name"], - scheduler=b["scheduler"], - positive=pos, - negative=neg, - latent_image=latent, - denoise=1.0 - ) - - trim_latent_val = trim_latent if trim_latent is not None else 0 - if trim_latent_val > 0: - latent_sampled["samples"] = latent_sampled["samples"][:, :, trim_latent_val:, :, :] - - try: - from .nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal - except ImportError: - raise ImportError("🚨 [XB-BOX] 找不到 _AliasVAEDecodeTemporal，请确保 nodes_vanilla_wrappers.py 未被删除！") - - cleanup_mode = b.get("cleanup", "不做任何清理") - decoder = XB_ROCmVAEDecodeTemporal() - decoded_image, = decoder.decode( - samples=latent_sampled, - vae=b["vae"], - tile=b.get("vae_decode_tile_size", 320), - overlap=b.get("spatial_overlap", 32), - t_tile=b.get("temporal_chunk_size", 64), - t_overlap=b.get("temporal_overlap", 8), - cleanup=cleanup_mode - ) - - is_4d = len(decoded_image.shape) == 4 - concat_dim = 0 if is_4d else 1 - - # 像素层裁切重叠过渡帧（trim_image 对应 continue_motion_max_frames） - trim_px = int(trim_image) if trim_image else 0 - if trim_px > 0 and trim_px < decoded_image.shape[concat_dim]: - if is_4d: - decoded_image = decoded_image[trim_px:, :, :, :] - else: - decoded_image = decoded_image[:, trim_px:, :, :, :] - print(f"✂️ [XB-BOX] 像素裁切重叠: {trim_px} 帧") - - final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="AnimateRelay") - - actual_out = decoded_image.shape[concat_dim] - b["current_offset"] = current_offset + actual_out - print(f"✅ [XB-BOX] Animate Relay Complete. trim_lat={trim_latent_val} trim_px={trim_px} out={actual_out}f total={final_video.shape[concat_dim]}f") - - return (b, final_video) - - -# ============================================================ -# XB_WanInfiniteTalk_ParamBus — InfiniteTalk 无限对口型参数总线 -# ============================================================ -class XB_WanInfiniteTalk_ParamBus: - """ - 集中管理所有共享参数。内置音频编码器 + CLIP 视觉编码。 - 接力点只需传正面提示词 + 分段帧数 + 音频，其余全从总线走。 - """ - _SCALE_METHODS = ["bilinear", "bicubic", "lanczos", "nearest-exact", "area"] - _CROP_MODES = ["center", "disabled"] - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - # --- 模型 --- - "model": ("MODEL",), - "model_patch": ("MODEL_PATCH",), - "clip": ("CLIP",), - "vae": ("VAE",), - "audio_encoder": ("AUDIO_ENCODER",), - # --- CLIP视觉 --- - "clip_vision": ("CLIP_VISION",), - # --- 基础参数 --- - "negative_prompt": ("STRING", {"multiline": True, "default": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"}), - "width": ("INT", {"default": 832, "min": 260, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "总线音频模式：音频切片节点输出的总帧数，耗尽后自动停止接力。0=独立音频模式"}), - "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), - # --- VAE 分块 --- - "vae_encode_tile_size": ("INT", {"default": 256, "min": 0, "max": 3840, "step": 32}), - "vae_decode_tile_size": ("INT", {"default": 192, "min": 0, "max": 3840, "step": 32}), - "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), - "temporal_chunk_size": ("INT", {"default": 96, "min": 0, "max": 8192, "step": 4}), - "temporal_overlap": ("INT", {"default": 4, "min": 0, "max": 8192, "step": 4}), - # --- 采样 --- - "steps": ("INT", {"default": 4, "min": 1, "max": 100}), - "cfg": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.1}), - "sampler_name": (comfy.samplers.KSAMPLER_NAMES,), - "scheduler": (comfy.samplers.SCHEDULER_NAMES,), - "seed": ("INT", {"default": 123456789}), - # --- InfiniteTalk --- - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), - "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), - # --- 缩放 --- - "scale_method": (cls._SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (cls._CROP_MODES, {"default": "center"}), - # --- 清理 --- - "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), - }, - "optional": { - "start_image": ("IMAGE",), - "audio": ("AUDIO", {"tooltip": "总线音频模式：接入长音频，总线一次性编码，接力点自动分段"}), - "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), - }, - } - - RETURN_TYPES = ("WAN_INFINITETALK_BUS",) - RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS",) - FUNCTION = "pack_bus" - CATEGORY = "XB_ToolBox/Pipeline" - - def pack_bus(self, **kwargs): - # CLIP 视觉编码：先缩放到视频尺寸，再按 crop 模式处理 - cv = kwargs.get("clip_vision") - ref = kwargs.get("start_image") - if cv is not None and ref is not None: - w, h = kwargs["width"], kwargs["height"] - method = kwargs.get("scale_method", "lanczos") - crop = kwargs.get("crop_mode", "center") - scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) - kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") - print(f"👁️ [XB-BOX] CLIP视觉编码: {ref.shape[1]}×{ref.shape[2]} → {w}×{h} ({method}/{crop})") - else: - kwargs["clip_vision_output"] = None - - # --- 总线音频模式：接入长音频 + 总帧数则一次性编码 --- - audio_input = kwargs.get("audio") - total_frames = kwargs.get("total_frames", 0) - if audio_input is not None and total_frames > 0: - ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") - if ae_cls is None: - raise ImportError("AudioEncoderEncode not found") - ae = ae_cls() - kwargs["_encoded_audio"], = getattr(ae, ae.FUNCTION)(kwargs["audio_encoder"], audio_input) - kwargs["_raw_audio"] = audio_input # 保留原始波形用于输出拼接 - kwargs["_total_frames"] = total_frames - kwargs["_accumulated_frames"] = 0 - kwargs["_bus_audio_mode"] = True - print(f"🔊 [XB-BOX] 总线音频模式：已编码长音频，总目标 {total_frames} 帧 ({total_frames/kwargs['fps']:.1f}s)") - else: - kwargs["_bus_audio_mode"] = False - - kwargs["_previous_frames"] = None - kwargs["_global_frame_offset"] = 0 - return (kwargs,) - - -# ============================================================ -# XB_WanInfiniteTalk_RelayNode — InfiniteTalk 无限对口型接力点 -# ============================================================ -class XB_WanInfiniteTalk_RelayNode: - """ - 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 - 音频独立输入 + 自动裁剪重叠 + 累加输出，与视频帧对齐。 - """ - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), - "positive_prompt": ("STRING", {"multiline": True, "default": ""}), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - }, - "optional": { - "prev_video": ("IMAGE",), - "prev_audio": ("AUDIO",), - "audio": ("AUDIO",), - } - } - - RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") - RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, - prev_video=None, prev_audio=None, audio=None): - b = wan_infinitetalk_bus.copy() - motion = b.get("motion_frame_count", 9) - fps = b.get("fps", 25.0) - bus_audio_mode = b.get("_bus_audio_mode", False) - - # ================================================================ - # 🛡️ 模式判断 & 帧数计算 - # ================================================================ - if bus_audio_mode: - # --- 模式1：总线音频模式 --- - total_frames = b["_total_frames"] - accumulated = b["_accumulated_frames"] - remaining = total_frames - accumulated - - if remaining <= 0: - print(f"\n⏭️ [XB-BOX] 总线帧数已耗尽 ({accumulated}/{total_frames})，跳过接力点") - fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) - fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} - return (b, fb_video, fb_audio) - - if remaining < segment_length: - # 最后一段：trim 会吃掉 motion 帧，需补回；再对齐 4N+1 - if accumulated == 0: - need_raw = remaining # 第一段无 trim - else: - need_raw = remaining + motion - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - print(f"⏭️ [XB-BOX] 剩余 {remaining} 帧不足以生成最小段，跳过") - fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) - fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} - return (b, fb_video, fb_audio) - print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - encoded_audio = b["_encoded_audio"] - raw_audio = b.get("_raw_audio") - use_segment_audio = False # 用 offset 方式裁剪长音频 - print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") - else: - # --- 模式2：独立音频模式 --- - if audio is None: - print("\n⏭️ [XB-BOX] 无音频输入，跳过接力点") - fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) - fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} - return (b, fb_video, fb_audio) - - actual_length = segment_length - encoded_audio = None # 下面编码 - raw_audio = audio - use_segment_audio = True # 每段独立音频从头开始 - print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") - - # --- 🛡️ 帧数对齐：Wan 模型要求 length = 4n+1 --- - raw_length = actual_length - aligned_length = ((actual_length + 2) // 4) * 4 + 1 - if aligned_length != raw_length: - print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") - # 总线模式下不补音频（已编码），独立模式下补静音 - if not bus_audio_mode and audio is not None: - pad_frames = aligned_length - raw_length - pad_sec = pad_frames / fps - sr = audio["sample_rate"] - pad_samples = int(pad_sec * sr) - pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) - raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} - actual_length = aligned_length - - # --- 音频编码（独立模式） --- - if not bus_audio_mode: - ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") - if ae_cls is None: - raise ImportError("AudioEncoderEncode not found") - ae = ae_cls() - encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) - - # --- 提示词 --- - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - # --- InfiniteTalk 底座 --- - prev = b.get("_previous_frames") - - # 🛡️ 总线模式：直接传递全局帧偏移量给底层，彻底废弃巨型 dummy tensor - # _process_infinite_talk_audio 现在接收 global_frame_offset 参数， - # 不再依赖 previous_frames.shape[0] 来反推音频偏移 - global_frame_offset = None - if bus_audio_mode: - global_frame_offset = b["_accumulated_frames"] - - core = XB_WanInfiniteTalkToVideo_Single() - - # � _process_infinite_talk_audio 内部已通过 model.clone() 保护原模型， - # 无需手动 save/restore——异常中断时 clone 自动被 GC 回收 - model_out, pos, neg, latent, trim = core.process( - model=b["model"], model_patch=b["model_patch"], - positive=pos_cond, negative=neg_cond, vae=b["vae"], - width=b["width"], height=b["height"], length=actual_length, - audio_encoder_output_1=encoded_audio, - motion_frame_count=motion, audio_scale=b["audio_scale"], - vae_tile_size=b["vae_encode_tile_size"], - clip_vision_output=b.get("clip_vision_output"), - start_image=b.get("start_image"), previous_frames=prev, - segment_audio=use_segment_audio, - global_frame_offset=global_frame_offset, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - # --- 采样 --- - latent_out, = nodes.KSampler().sample( - model=model_out, seed=b["seed"], - steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0, - ) - - # --- 解码 --- - decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) - - # --- 视频裁剪累加 --- - is4d = len(decoded.shape) == 4 - cd = 0 if is4d else 1 - total = decoded.shape[cd] - b["_previous_frames"] = decoded - - trim_val = int(trim) if trim else 0 - trim_val = min(trim_val, total - 1) if trim_val < total else 0 - cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] - final_video = _safe_video_accumulate(prev_video, cut, cd, label="TalkRelay") - - # --- 音频裁剪累加 --- - ats = trim_val / fps if trim_val else 0.0 - if bus_audio_mode: - # 总线模式：从原始长音频中裁出当前段 - raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) - raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 - seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) - seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) - seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) - cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None - else: - cut_audio = raw_audio - if ats > 0 and raw_audio: - wf = raw_audio["waveform"] - sr = raw_audio["sample_rate"] - n = int(ats * sr) - if n < wf.shape[-1]: - cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} - else: - cut_audio = None # 裁切长度超出自身，彻底丢弃防止音画不同步 - - if prev_audio is None: - final_audio = cut_audio - elif cut_audio is None: - final_audio = prev_audio - else: - pa, ca = prev_audio, cut_audio - if pa["sample_rate"] != ca["sample_rate"]: - if torchaudio is None: - raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") - ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) - ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} - - # 🛡️ 设备对齐 + 声道对齐: 防止 torch.cat 因设备/声道不一致崩溃 - pa_wf = pa["waveform"] - ca_wf = ca["waveform"].to(pa_wf.device) - if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: - # 🔧 严格针对 Channel 维度 (shape[-2]) 对齐 [B, C, T] 格式 - if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: - rpt = [1] * pa_wf.dim() - rpt[-2] = ca_wf.shape[-2] - pa_wf = pa_wf.repeat(*rpt) - elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: - rpt = [1] * ca_wf.dim() - rpt[-2] = pa_wf.shape[-2] - ca_wf = ca_wf.repeat(*rpt) - - final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), - "sample_rate": pa["sample_rate"]} - - b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] - if bus_audio_mode: - b["_accumulated_frames"] += cut.shape[cd] - remaining_after = b["_total_frames"] - b["_accumulated_frames"] - print(f"🎵 [XB-BOX] Relay done: {total}→{cut.shape[cd]}f (trim {trim_val}), " - f"acc={final_video.shape[cd]}f, 总线进度 {b['_accumulated_frames']}/{b['_total_frames']} (剩余 {remaining_after})") - else: - print(f"🎵 [XB-BOX] Relay done: {total}→{cut.shape[cd]}f (trim {trim_val}), " - f"acc={final_video.shape[cd]}f, video_dur={final_video.shape[cd]/fps:.1f}s") - - _refresh_models() - return (b, final_video, final_audio) - -# ============================================================ -# NEW 无限接力节点（带重叠帧数/拼接模式等新功能） -# ============================================================ - -class XB_Wan_InfiniteRelayNode_New: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "wan_bus": ("WAN_BUS",), - "start_image": ("IMAGE",), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), - "trim_head_frames": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1, "tooltip": "接力重叠帧数（去重）"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定：本节点自动循环 N 次 = 串联 N 个接力点"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",) - } - } - - @classmethod - def VALIDATE_INPUTS(cls, **kwargs): - return True - - RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") - RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_bus, start_image, positive_prompt, trim_head_frames=1, relay_count=1, total_frames_display="", prev_video=None): - print(f"\n🏃‍♀️ [XB-BOX] Executing Image-to-Video Infinite pipeline × {relay_count} relay(s)...") - b = wan_bus - segment_len = b["length"] - cut_first_frame = b.get("cut_first_frame", True) - # 估算总计帧数 - est_per_segment = segment_len - (trim_head_frames if cut_first_frame and segment_len > 1 else 0) - est_total = est_per_segment * relay_count - print(f"📊 [XB-BOX] 每段约 {est_per_segment} 帧 (原始 {segment_len}), 总计约 {est_total} 帧") - - current_image = start_image - accumulated_video = prev_video - - for r in range(relay_count): - seed = b["seed"] - print(f"\n🔄 [XB-BOX] 接力 {r+1}/{relay_count}...") - - encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) - decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) - - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - cv_output = b.get("clip_vision_output") - pos, neg, latent = XB_WanImageToVideo().process( - positive=pos_cond, negative=neg_cond, vae=b["vae"], - clip_vision_output=cv_output, start_image=current_image, - width=b["width"], height=b["height"], length=segment_len, - batch_size=1, vae_tile_size=encode_tile, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - total_steps = b["steps"] - high_steps = b["high_noise_steps"] - if high_steps >= total_steps: - high_steps = total_steps - 1 - - latent_high, = nodes.KSamplerAdvanced().sample( - model=b["model_high"], add_noise="enable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, - start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" - ) - - latent_low, = nodes.KSamplerAdvanced().sample( - model=b["model_low"], add_noise="disable", noise_seed=seed, - steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent_high, - start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" - ) - - decoded_image, = nodes.VAEDecode().decode(samples=latent_low, vae=b["vae"]) - - is_4d = len(decoded_image.shape) == 4 - frame_dim = 0 if is_4d else 1 - total_frames = decoded_image.shape[frame_dim] - - if cut_first_frame and total_frames > 1: - safe_trim = min(trim_head_frames, total_frames - 1) - if safe_trim > 0: - if is_4d: - decoded_image = decoded_image[safe_trim:, :, :, :] - else: - decoded_image = decoded_image[:, safe_trim:, :, :, :] - - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - - decoded_image = _apply_progressive_color_correction(decoded_image, current_image, is_4d) - last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] - - concat_dim = 0 if is_4d else 1 - accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"InfiniteRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) - current_image = last_frame - - _refresh_models() - print(f"✅ [XB-BOX] 接力完成: {relay_count} 段, 总计 {accumulated_video.shape[concat_dim]} 帧") - return (wan_bus, current_image, accumulated_video) - - -# ============================================================ -# XB_Video_Merger — 视频拼接器 -# ============================================================ -class XB_Video_Merger: - @classmethod - def INPUT_TYPES(cls): - inputs = {"required": {}, "optional": {}} - for i in range(1, 11): - inputs["optional"][f"video_{i}"] = ("IMAGE",) - return inputs - - RETURN_TYPES = ("IMAGE",) - RETURN_NAMES = ("🎞️ Full Long Video",) - FUNCTION = "merge_videos" - CATEGORY = "XB_ToolBox/Pipeline" - - def merge_videos(self, **kwargs): - videos = [kwargs.get(f"video_{i}") for i in range(1, 11) if kwargs.get(f"video_{i}") is not None] - if not videos: - raise ValueError("🚨 [XB-BOX] Merge failed: At least one video must be connected!") - - concat_dim = 0 if len(videos[0].shape) == 4 else 1 - ref_shape = list(videos[0].shape) - for i, vid in enumerate(videos): - for d in range(len(ref_shape)): - if d != concat_dim and vid.shape[d] != ref_shape[d]: - raise ValueError( - f"🚨 [XB-BOX] Merge failed: Video segment {i+1} shape {list(vid.shape)} " - f"does not match first segment shape {ref_shape} on dimension {d} " - f"(expected {ref_shape[d]}, got {vid.shape[d]})" - ) - # 将全部视频段卸载到 CPU 再拼接，防止多段同时驻留 VRAM 导致 OOM - videos_cpu = [v.cpu() for v in videos] - merged = torch.cat(videos_cpu, dim=concat_dim) - print(f"💾 [XB-BOX] Merger: {len(videos)} 段视频已在 CPU 完成拼接 (shape={list(merged.shape)})") - return (merged,) - -# ============================================================ -# XB_StoryboardSlicer — 分镜切片器 -# ============================================================ -class XB_StoryboardSlicer: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "image": ("IMAGE",), - "mode": (["3-Grid (1x3 Horizontal)", "3-Grid (3x1 Vertical)", "4-Grid (2x2)", "6-Grid (2x3 Horizontal)", "6-Grid (3x2 Vertical)", "9-Grid (3x3)"], {"default": "4-Grid (2x2)"}), - "crop_margin": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 0.45, "step": 0.01}), - } - } - RETURN_TYPES = ("IMAGE",) * 9 - RETURN_NAMES = ("Img1", "Img2", "Img3", "Img4", "Img5", "Img6", "Img7", "Img8", "Img9") - FUNCTION = "slice_image" - CATEGORY = "XB_ToolBox/Pipeline" - - def slice_image(self, image, mode, crop_margin): - if "1x3" in mode: rows, cols = 1, 3 - elif "3x1" in mode: rows, cols = 3, 1 - elif "2x2" in mode: rows, cols = 2, 2 - elif "2x3" in mode: rows, cols = 2, 3 - elif "3x2" in mode: rows, cols = 3, 2 - elif "3x3" in mode: rows, cols = 3, 3 - else: rows, cols = 2, 2 - - batch_size, h, w, channels = image.shape - slice_h, slice_w = h // rows, w // cols - crop_y, crop_x = int(slice_h * crop_margin), int(slice_w * crop_margin) - - sliced_images = [] - for r in range(rows): - for c in range(cols): - sy, ey = r * slice_h + crop_y, (r + 1) * slice_h - crop_y - sx, ex = c * slice_w + crop_x, (c + 1) * slice_w - crop_x - if sy >= ey: ey = sy + 1 - if sx >= ex: ex = sx + 1 - sliced_images.append(image[:, sy:ey, sx:ex, :]) - - placeholder_h = max(1, slice_h - crop_y * 2) - placeholder_w = max(1, slice_w - crop_x * 2) - while len(sliced_images) < 9: - sliced_images.append(torch.zeros((1, placeholder_h, placeholder_w, 3), dtype=torch.float32, device=image.device)) - return tuple(sliced_images) - - -# ============================================================ -# XB_WanAnimate_ParamBus — Wan Animate 动作迁移参数总线 -# ============================================================ -class XB_WanAnimate_RelayNode_New: - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] - files = sorted(files) - - if not files: - files = ["[Folder empty_Please connect or upload]"] - - return { - "required": { - "wan_animate_bus": ("WAN_ANIMATE_BUS",), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), - "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), - "ref_image_file": (files, {"image_upload": True}), - "continue_motion_max_frames": ("INT", {"default": 5, "min": 0, "max": 16, "tooltip": "接力重叠帧数（衔接过渡）"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",), - } - } - - @classmethod - def VALIDATE_INPUTS(cls, **kwargs): - return True - - RETURN_TYPES = ("WAN_ANIMATE_BUS", "IMAGE") - RETURN_NAMES = ("📦 WAN_ANIMATE_BUS (传给下段)", "🎞️ 累加视频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_animate_bus, segment_length, positive_prompt="", use_local_ref_image="继承总线全局图", ref_image_file="", continue_motion_max_frames=5, relay_count=1, total_frames_display="", prev_video=None, opt_ref_image=None): - b = wan_animate_bus.copy() - print(f"\n🏃‍♀️ [XB-BOX] Executing Wan Animate Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") - accumulated_video = prev_video - - completed = 0 - for r in range(relay_count): - current_offset = b.get("current_offset", 0) - seed = b.get("seed", 123456789) - print(f"\n🔄 [XB-BOX] Animate 接力 {r+1}/{relay_count} (offset={current_offset})...") - - source_video = b.get("pose_video") if b.get("pose_video") is not None else b.get("face_video") - total_source_frames = float('inf') - bus_total = b.get("total_frames", 0) - if bus_total > 0: - total_source_frames = bus_total - elif source_video is not None: - total_source_frames = source_video.shape[0] if len(source_video.shape) == 4 else source_video.shape[1] - - remaining_frames = max(0, total_source_frames - current_offset) - - if remaining_frames <= 0: - break - - if remaining_frames < segment_length: - if current_offset == 0: - need_raw = remaining_frames - else: - need_raw = remaining_frames + continue_motion_max_frames - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - break - print(f"⚠️ [XB-BOX] 最后一段: 剩余 {remaining_frames} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - is_local = (use_local_ref_image in ("独立参考图", True, 1)) - if is_local: - if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": - print("⏭️ [XB-BOX] 独立参考图未选择，跳过") - break - image_path = folder_paths.get_annotated_filepath(ref_image_file) - if not os.path.exists(image_path): - print(f"⏭️ [XB-BOX] 参考图文件未找到，跳过") - break - i = Image.open(image_path) - i = ImageOps.exif_transpose(i) - image_data = i.convert("RGB") - image_data = np.array(image_data).astype(np.float32) / 255.0 - ref_image = torch.from_numpy(image_data)[None,] - cv_model = b.get("clip_vision") - if cv_model is None: - raise ValueError("🚨 独立参考图需要 clip_vision 模型！") - w, h = b.get("width", 480), b.get("height", 832) - sm = b.get("scale_method", "lanczos") - cm = b.get("crop_mode", "center") - clip_img = comfy.utils.common_upscale(ref_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) - clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") - else: - ref_image = b.get("global_ref_image") - if ref_image is None: - raise ValueError("🚨 缺少参考图！") - clip_vision_output = b.get("clip_vision_output") - - prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) - - continue_motion = None - cont_max_frames = continue_motion_max_frames - if is_local: - cont_max_frames = 0 - if accumulated_video is not None and cont_max_frames > 0: - is_4d_v = len(accumulated_video.shape) == 4 - if is_4d_v: - continue_motion = accumulated_video[-cont_max_frames:, :, :, :] - else: - continue_motion = accumulated_video[:, -cont_max_frames:, :, :, :] - - try: - from .nodes_wan import XB_WanAnimateToVideo - except ImportError: - raise ImportError("🚨 找不到 XB_WanAnimateToVideo") - - animate_core = XB_WanAnimateToVideo() - func_name = getattr(XB_WanAnimateToVideo, "FUNCTION", "process") - func = getattr(animate_core, func_name) - kwargs = { - "positive": pos_cond, "negative": neg_cond, "vae": b["vae"], - "clip_vision_output": clip_vision_output, "reference_image": ref_image, - "face_video": b.get("face_video"), "pose_video": b.get("pose_video"), - "background_video": b.get("background_video"), "character_mask": b.get("character_mask"), - "continue_motion": continue_motion, - "width": b["width"], "height": b["height"], "length": actual_length, "batch_size": 1, - "continue_motion_max_frames": cont_max_frames, - "video_frame_offset": current_offset, - "vae_tile_size": b.get("vae_encode_tile_size", 320) - } - pos, neg, latent, trim_latent, trim_image, new_offset = func(**kwargs) - - latent_sampled, = nodes.KSampler().sample( - model=b["model"], seed=seed, steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0 - ) - trim_latent_val = trim_latent if trim_latent is not None else 0 - if trim_latent_val > 0: - latent_sampled["samples"] = latent_sampled["samples"][:, :, trim_latent_val:, :, :] - - try: - from .nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal - except ImportError: - raise ImportError("🚨 找不到 _AliasVAEDecodeTemporal") - cleanup_mode = b.get("cleanup", "不做任何清理") - decoder = XB_ROCmVAEDecodeTemporal() - decoded_image, = decoder.decode( - samples=latent_sampled, vae=b["vae"], - tile=b.get("vae_decode_tile_size", 320), overlap=b.get("spatial_overlap", 32), - t_tile=b.get("temporal_chunk_size", 64), t_overlap=b.get("temporal_overlap", 8), - cleanup=cleanup_mode - ) - - is_4d = len(decoded_image.shape) == 4 - concat_dim = 0 if is_4d else 1 - trim_px = int(trim_image) if trim_image else 0 - if trim_px > 0 and trim_px < decoded_image.shape[concat_dim]: - if is_4d: - decoded_image = decoded_image[trim_px:, :, :, :] - else: - decoded_image = decoded_image[:, trim_px:, :, :, :] - - accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"AnimateRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) - actual_out = decoded_image.shape[concat_dim] - b["current_offset"] = current_offset + actual_out - print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={actual_out}f total={accumulated_video.shape[concat_dim]}f") - completed = r + 1 - - if completed < relay_count: - print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") - else: - print(f"🎉 [XB-BOX] Animate 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") - overlap = continue_motion_max_frames - total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) - return (b, accumulated_video) - - -# ============================================================ -# XB_WanInfiniteTalk_ParamBus — InfiniteTalk 无限对口型参数总线 -# ============================================================ -class XB_WanInfiniteTalk_RelayNode_New: - """ - 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 - 音频独立输入 + 自动裁剪重叠 + 累加输出，与视频帧对齐。 - relay_count > 1 时本节点内部自动循环 N 次。 - """ - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), - "positive_prompt": ("STRING", {"multiline": True, "default": ""}), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",), - "prev_audio": ("AUDIO",), - "audio": ("AUDIO",), - } - } - - RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") - RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, motion_frame_count=9, relay_count=1, total_frames_display="", - prev_video=None, prev_audio=None, audio=None): - b = wan_infinitetalk_bus.copy() - fps = b.get("fps", 25.0) - bus_audio_mode = b.get("_bus_audio_mode", False) - print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") - - accumulated_video = prev_video - accumulated_audio = prev_audio - - completed = 0 - for r in range(relay_count): - print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") - - # ================================================================ - # 🛡️ 模式判断 & 帧数计算 - # ================================================================ - if bus_audio_mode: - # --- 模式1：总线音频模式 --- - total_frames = b["_total_frames"] - accumulated = b["_accumulated_frames"] - remaining = total_frames - accumulated - - if remaining <= 0: - break - - if remaining < segment_length: - if accumulated == 0: - need_raw = remaining # 第一段无 trim - else: - need_raw = remaining + motion_frame_count - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - break - print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - encoded_audio = b["_encoded_audio"] - raw_audio = b.get("_raw_audio") - use_segment_audio = False - print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") - else: - # --- 模式2：独立音频模式 --- - if audio is None: - print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") - break - - actual_length = segment_length - encoded_audio = None - raw_audio = audio - use_segment_audio = True - print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") - - # --- 🛡️ 帧数对齐：Wan 模型要求 length = 4n+1 --- - raw_length = actual_length - aligned_length = ((actual_length + 2) // 4) * 4 + 1 - if aligned_length != raw_length: - print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") - if not bus_audio_mode and audio is not None: - pad_frames = aligned_length - raw_length - pad_sec = pad_frames / fps - sr = audio["sample_rate"] - pad_samples = int(pad_sec * sr) - pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) - raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} - actual_length = aligned_length - - # --- 音频编码（独立模式） --- - if not bus_audio_mode: - ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") - if ae_cls is None: - raise ImportError("AudioEncoderEncode not found") - ae = ae_cls() - encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) - - # --- 提示词 --- - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - # --- InfiniteTalk 底座 --- - prev = b.get("_previous_frames") - global_frame_offset = None - if bus_audio_mode: - global_frame_offset = b["_accumulated_frames"] - - core = XB_WanInfiniteTalkToVideo_Single() - model_out, pos, neg, latent, trim = core.process( - model=b["model"], model_patch=b["model_patch"], - positive=pos_cond, negative=neg_cond, vae=b["vae"], - width=b["width"], height=b["height"], length=actual_length, - audio_encoder_output_1=encoded_audio, - motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], - vae_tile_size=b["vae_encode_tile_size"], - clip_vision_output=b.get("clip_vision_output"), - start_image=b.get("start_image"), previous_frames=prev, - segment_audio=use_segment_audio, - global_frame_offset=global_frame_offset, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - # --- 采样 --- - latent_out, = nodes.KSampler().sample( - model=model_out, seed=b["seed"], - steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0, - ) - - # --- 解码 --- - decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) - - # --- 视频裁剪累加 --- - is4d = len(decoded.shape) == 4 - cd = 0 if is4d else 1 - total = decoded.shape[cd] - b["_previous_frames"] = decoded - - trim_val = int(trim) if trim else 0 - trim_val = min(trim_val, total - 1) if trim_val < total else 0 - cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] - final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) - - # --- 音频裁剪累加 --- - ats = trim_val / fps if trim_val else 0.0 - if bus_audio_mode: - raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) - raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 - seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) - seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) - seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) - cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None - else: - cut_audio = raw_audio - if ats > 0 and raw_audio: - wf = raw_audio["waveform"] - sr = raw_audio["sample_rate"] - n = int(ats * sr) - if n < wf.shape[-1]: - cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} - else: - cut_audio = None - - if accumulated_audio is None: - final_audio = cut_audio - elif cut_audio is None: - final_audio = accumulated_audio - else: - pa, ca = accumulated_audio, cut_audio - if pa["sample_rate"] != ca["sample_rate"]: - if torchaudio is None: - raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") - ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) - ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} - pa_wf = pa["waveform"] - ca_wf = ca["waveform"].to(pa_wf.device) - if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: - if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: - rpt = [1] * pa_wf.dim() - rpt[-2] = ca_wf.shape[-2] - pa_wf = pa_wf.repeat(*rpt) - elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: - rpt = [1] * ca_wf.dim() - rpt[-2] = pa_wf.shape[-2] - ca_wf = ca_wf.repeat(*rpt) - final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} - - b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] - if bus_audio_mode: - b["_accumulated_frames"] += cut.shape[cd] - accumulated_video = final_video - accumulated_audio = final_audio - print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f total={final_video.shape[cd]}f") - completed = r + 1 - - if completed < relay_count: - print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") - else: - print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") - _refresh_models() - overlap = motion_frame_count - total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) - return (b, accumulated_video, accumulated_audio) - - -# ============================================================ -# XB_WanSCAIL_ParamBus — SCAIL 无限时长参数总线 -# ============================================================ -class XB_WanSCAIL_ParamBus_New: - """SCAIL 无限接力参数总线。 - 集中管理模型、姿态视频、参考图等所有共享参数， - 接力点只需传正面提示词 + 单次生成帧数，其余全从总线走。 - """ - _SCALE_METHODS = ["bilinear", "bicubic", "lanczos", "nearest-exact", "area"] - _CROP_MODES = ["center", "disabled"] - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - # --- 模型 --- - "model": ("MODEL",), - "clip": ("CLIP",), - "vae": ("VAE",), - "clip_vision": ("CLIP_VISION",), - # --- 基础参数 --- - "negative_prompt": ("STRING", {"multiline": True, "default": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"}), - "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), - "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), - "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "0=自动从姿态视频取长度；>0=强制截断到此帧数"}), - "fps": ("FLOAT", {"default": 16.0, "min": 1.0, "max": 120.0, "step": 1.0}), - # --- SCAIL 控制 --- - "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), - "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "replacement_mode": ("BOOLEAN", {"default": False, "label_on": "替换模式", "label_off": "动画模式"}), - "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置: 自动=显存>30%切CPU | 强制GPU=始终GPU | 强制CPU=始终CPU"}), - # --- VAE 分块 --- - "vae_encode_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - "vae_decode_tile_size": ("INT", {"default": 192, "min": 64, "max": 3840, "step": 32}), - "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), - "temporal_chunk_size": ("INT", {"default": 64, "min": 0, "max": 8192, "step": 4}), - "temporal_overlap": ("INT", {"default": 8, "min": 0, "max": 8192, "step": 4}), - # --- 采样 --- - "steps": ("INT", {"default": 20, "min": 1, "max": 100}), - "cfg": ("FLOAT", {"default": 5.0, "min": 0.0, "max": 10.0, "step": 0.1}), - "sampler_name": (comfy.samplers.KSAMPLER_NAMES,), - "scheduler": (comfy.samplers.SCHEDULER_NAMES,), - "seed": ("INT", {"default": 123456789}), - # --- 清理 --- - "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), - }, - "optional": { - "global_ref_image": ("IMAGE",), - "reference_image_mask": ("IMAGE", {"tooltip": "参考图遮罩 (替换模式: 隔离角色区域)"}), - "pose_video": ("IMAGE", {"tooltip": "姿态/驱动视频"}), - "pose_video_mask": ("IMAGE", {"tooltip": "SCAIL-2 彩色遮罩视频"}), - "scale_method": (cls._SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (cls._CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("WAN_SCAIL_BUS",) - RETURN_NAMES = ("📦 WAN_SCAIL_BUS",) - FUNCTION = "pack_bus" - CATEGORY = "XB_ToolBox/Pipeline" - - def pack_bus(self, **kwargs): - # CLIP 视觉编码：参考图缩放到视频尺寸后编码 - cv = kwargs.get("clip_vision") - ref = kwargs.get("global_ref_image") - if cv is not None and ref is not None: - w, h = kwargs["width"], kwargs["height"] - method = kwargs.get("scale_method", "lanczos") - crop = kwargs.get("crop_mode", "center") - scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) - kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") - print(f"👁️ [XB-BOX] SCAIL CLIP视觉编码 ({ref.shape[1]}×{ref.shape[2]} → {w}×{h}, {method}/{crop})") - else: - kwargs["clip_vision_output"] = None - - kwargs["current_offset"] = 0 - return (kwargs,) - - -# ============================================================ -# XB_WanSCAIL_RelayNode — SCAIL 无限接力点 -# ============================================================ -class XB_WanSCAIL_RelayNode_New: - """SCAIL 无限接力节点。 - 每个接力点完成：CLIPTextEncode → SCAIL Process → KSampler → VAEDecode → trim → 累加。 - 自动管理姿态视频偏移 + 前段尾帧衔接。 - relay_count > 1 时本节点内部自动循环 N 次。 - """ - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "wan_scail_bus": ("WAN_SCAIL_BUS",), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), - "previous_frame_count": ("INT", {"default": 5, "min": 1, "max": 8192, "step": 4, "tooltip": "接力重叠帧数（尾帧衔接）"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",), - } - } - - RETURN_TYPES = ("WAN_SCAIL_BUS", "IMAGE") - RETURN_NAMES = ("📦 WAN_SCAIL_BUS (传给下段)", "🎞️ 累加视频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_scail_bus, segment_length, positive_prompt="", previous_frame_count=5, relay_count=1, total_frames_display="", prev_video=None): - b = wan_scail_bus.copy() - print(f"\n🏃‍♀️ [XB-BOX] Executing SCAIL Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") - accumulated_video = prev_video - - completed = 0 - for r in range(relay_count): - current_offset = b.get("current_offset", 0) - seed = b.get("seed", 123456789) - print(f"\n🔄 [XB-BOX] SCAIL 接力 {r+1}/{relay_count} (offset={current_offset})...") - - pose_video = b.get("pose_video") - total_source_frames = float('inf') - bus_total = b.get("total_frames", 0) - if bus_total > 0: - total_source_frames = bus_total - elif pose_video is not None: - total_source_frames = pose_video.shape[0] if len(pose_video.shape) == 4 else pose_video.shape[1] - - remaining_frames = max(0, total_source_frames - current_offset) - - if remaining_frames <= 0: - break - - if remaining_frames < segment_length: - if current_offset == 0: - need_raw = remaining_frames - else: - need_raw = remaining_frames + previous_frame_count - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - break - print(f"⚠️ [XB-BOX] SCAIL 最后一段: 剩余 {remaining_frames} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - # --- 参考图：始终从总线继承（已含遮罩处理） --- - ref_image = b.get("global_ref_image") - clip_vision_output = b.get("clip_vision_output") - - # --- 提示词 --- - prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) - - # --- 准备 previous_frames --- - previous_frames = None - if accumulated_video is not None and current_offset > 0: - is_4d = len(accumulated_video.shape) == 4 - if is_4d: - previous_frames = accumulated_video[-previous_frame_count:] - else: - previous_frames = accumulated_video[:, -previous_frame_count:] - - # --- SCAIL Pro 底座 --- - try: - from .nodes_wan_vae import XB_WanSCAILToVideoPro - except ImportError: - raise ImportError("🚨 找不到 XB_WanSCAILToVideoPro") - scail_core = XB_WanSCAILToVideoPro() - pos, neg, latent, new_offset = scail_core.process( - positive=pos_cond, negative=neg_cond, vae=b["vae"], - width=b["width"], height=b["height"], length=actual_length, batch_size=1, - pose_strength=b.get("pose_strength", 1.0), - pose_start=b.get("pose_start", 0.0), pose_end=b.get("pose_end", 1.0), - replacement_mode=b.get("replacement_mode", False), - video_frame_offset=current_offset, previous_frame_count=previous_frame_count, - vae_tile_size=b.get("vae_encode_tile_size", 256), - pose_video=pose_video, pose_video_mask=b.get("pose_video_mask"), - reference_image=ref_image, reference_image_mask=b.get("reference_image_mask"), - clip_vision_output=clip_vision_output, previous_frames=previous_frames, - ) - - # --- 采样 --- - latent_sampled, = nodes.KSampler().sample( - model=b["model"], seed=seed, steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0, - ) - - # --- 解码 --- - try: - from .nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal - except ImportError: - raise ImportError("🚨 找不到 _AliasVAEDecodeTemporal") - cleanup_mode = b.get("cleanup", "不做任何清理") - decoder = XB_ROCmVAEDecodeTemporal() - decoded_image, = decoder.decode( - samples=latent_sampled, vae=b["vae"], - tile=b.get("vae_decode_tile_size", 192), overlap=b.get("spatial_overlap", 32), - t_tile=b.get("temporal_chunk_size", 64), t_overlap=b.get("temporal_overlap", 8), - cleanup=cleanup_mode, - ) - - # --- 裁剪重叠帧 & 累加 --- - is_4d = len(decoded_image.shape) == 4 - concat_dim = 0 if is_4d else 1 - total_out = decoded_image.shape[concat_dim] - trim_px = previous_frame_count if (accumulated_video is not None and current_offset > 0) else 0 - trim_px = min(trim_px, total_out - 1) if trim_px < total_out else 0 - if trim_px > 0: - if is_4d: - decoded_image = decoded_image[trim_px:, :, :, :] - else: - decoded_image = decoded_image[:, trim_px:, :, :, :] - - accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"SCAIL-{r+1}", concat_mode=b.get("concat_mode", "自动")) - actual_out = decoded_image.shape[concat_dim] - b["current_offset"] = new_offset - print(f"✅ [XB-BOX] SCAIL 接力 {r+1}/{relay_count} 完成. out={actual_out}f total={accumulated_video.shape[concat_dim]}f") - completed = r + 1 - - if completed < relay_count: - print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") - else: - print(f"🎉 [XB-BOX] SCAIL 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") - _refresh_models() - overlap = previous_frame_count - total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) - return (b, accumulated_video) - - -# ============================================================ -# XB_WanInfiniteTalk_RelayNode_MultiRef — InfiniteTalk 多图接力点 -# ============================================================ -class XB_WanInfiniteTalk_RelayNode_MultiRef: - """ - InfiniteTalk 无限对口型接力点（多图版）。 - 支持独立参考图上传，类似 Animate 接力点。 - 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 - relay_count > 1 时本节点内部自动循环 N 次。 - """ - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] - files = sorted(files) - if not files: - files = ["[Folder empty_Please connect or upload]"] - - return { - "required": { - "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), - "positive_prompt": ("STRING", {"multiline": True, "default": ""}), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), - "ref_image_file": (files, {"image_upload": True}), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",), - "prev_audio": ("AUDIO",), - "audio": ("AUDIO",), - } - } - - RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") - RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, use_local_ref_image="继承总线全局图", ref_image_file="", - motion_frame_count=9, relay_count=1, total_frames_display="", - prev_video=None, prev_audio=None, audio=None): - b = wan_infinitetalk_bus.copy() - fps = b.get("fps", 25.0) - bus_audio_mode = b.get("_bus_audio_mode", False) - print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk MultiRef Relay × {relay_count} relay(s)... 每段 {segment_length} 帧") - - accumulated_video = prev_video - accumulated_audio = prev_audio - - # --- 预处理独立参考图 --- - is_local = (use_local_ref_image in ("独立参考图", True, 1)) - local_start_image = None - local_clip_vision_output = None - if is_local: - if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": - raise ValueError("🚨 [XB-BOX] 多图接力点已开启独立参考图，但未选择图片文件！") - image_path = folder_paths.get_annotated_filepath(ref_image_file) - if not os.path.exists(image_path): - raise ValueError(f"🚨 [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}") - i = Image.open(image_path) - i = ImageOps.exif_transpose(i) - image_data = i.convert("RGB") - image_data = np.array(image_data).astype(np.float32) / 255.0 - local_start_image = torch.from_numpy(image_data)[None,] - cv_model = b.get("clip_vision") - if cv_model is None: - raise ValueError("🚨 [XB-BOX] 独立参考图需要总线连接 clip_vision 模型！") - w, h = b["width"], b["height"] - sm = b.get("scale_method", "lanczos") - cm = b.get("crop_mode", "center") - clip_img = comfy.utils.common_upscale(local_start_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) - local_clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") - print(f"🖼️ [XB-BOX] 多图接力点使用独立参考图 ({image_data.shape[1]}×{image_data.shape[0]} → {w}×{h}, {sm}/{cm})") - - completed = 0 - for r in range(relay_count): - print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") - - if bus_audio_mode: - total_frames = b["_total_frames"] - accumulated = b["_accumulated_frames"] - remaining = total_frames - accumulated - - if remaining <= 0: - break - - if remaining < segment_length: - if accumulated == 0: - need_raw = remaining - else: - need_raw = remaining + motion_frame_count - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - break - print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - encoded_audio = b["_encoded_audio"] - raw_audio = b.get("_raw_audio") - use_segment_audio = False - print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") - else: - if audio is None: - print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") - break - - actual_length = segment_length - encoded_audio = None - raw_audio = audio - use_segment_audio = True - print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") - - # --- 帧数对齐 4N+1 --- - raw_length = actual_length - aligned_length = ((actual_length + 2) // 4) * 4 + 1 - if aligned_length != raw_length: - print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") - if not bus_audio_mode and audio is not None: - pad_frames = aligned_length - raw_length - pad_sec = pad_frames / fps - sr = audio["sample_rate"] - pad_samples = int(pad_sec * sr) - pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) - raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} - actual_length = aligned_length - - # --- 音频编码（独立模式） --- - if not bus_audio_mode: - ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") - if ae_cls is None: - raise ImportError("AudioEncoderEncode not found") - ae = ae_cls() - encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) - - # --- 提示词 --- - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - # --- 参考图/CLIP视觉 --- - prev = b.get("_previous_frames") - global_frame_offset = None - if bus_audio_mode: - global_frame_offset = b["_accumulated_frames"] - - # 独立参考图：每个接力段使用独立编码；总线参考图：从总线获取 - if is_local: - cur_start_image = local_start_image - cur_clip_output = local_clip_vision_output - else: - cur_start_image = b.get("start_image") - cur_clip_output = b.get("clip_vision_output") - - # --- InfiniteTalk 底座 --- - core = XB_WanInfiniteTalkToVideo_Single() - model_out, pos, neg, latent, trim = core.process( - model=b["model"], model_patch=b["model_patch"], - positive=pos_cond, negative=neg_cond, vae=b["vae"], - width=b["width"], height=b["height"], length=actual_length, - audio_encoder_output_1=encoded_audio, - motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], - vae_tile_size=b["vae_encode_tile_size"], - clip_vision_output=cur_clip_output, - start_image=cur_start_image, previous_frames=prev, - segment_audio=use_segment_audio, - global_frame_offset=global_frame_offset, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - # --- 采样 --- - latent_out, = nodes.KSampler().sample( - model=model_out, seed=b["seed"], - steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0, - ) - - # --- 解码 --- - decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) - - # --- 视频裁剪累加 --- - is4d = len(decoded.shape) == 4 - cd = 0 if is4d else 1 - total = decoded.shape[cd] - b["_previous_frames"] = decoded - - trim_val = int(trim) if trim else 0 - trim_val = min(trim_val, total - 1) if trim_val < total else 0 - cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] - final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkMultiRef-{r+1}", concat_mode=b.get("concat_mode", "自动")) - - # --- 音频裁剪累加 --- - ats = trim_val / fps if trim_val else 0.0 - if bus_audio_mode: - raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) - raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 - seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) - seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) - seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) - cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None - else: - cut_audio = raw_audio - if ats > 0 and raw_audio: - wf = raw_audio["waveform"] - sr = raw_audio["sample_rate"] - n = int(ats * sr) - if n < wf.shape[-1]: - cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} - else: - cut_audio = None - - if accumulated_audio is None: - final_audio = cut_audio - elif cut_audio is None: - final_audio = accumulated_audio - else: - pa, ca = accumulated_audio, cut_audio - if pa["sample_rate"] != ca["sample_rate"]: - if torchaudio is None: - raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") - ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) - ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} - pa_wf = pa["waveform"] - ca_wf = ca["waveform"].to(pa_wf.device) - if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: - if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: - rpt = [1] * pa_wf.dim(); rpt[-2] = ca_wf.shape[-2]; pa_wf = pa_wf.repeat(*rpt) - elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: - rpt = [1] * ca_wf.dim(); rpt[-2] = pa_wf.shape[-2]; ca_wf = ca_wf.repeat(*rpt) - final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} - - b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] - if bus_audio_mode: - b["_accumulated_frames"] += cut.shape[cd] - accumulated_video = final_video - accumulated_audio = final_audio - print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f total={final_video.shape[cd]}f") - completed = r + 1 - - if completed < relay_count: - print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") - else: - print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") - # 👑 刷新总线全局状态：如果引入了独立参考图，将其升级为下游节点的新基准 - if is_local_ref: - b["start_image"] = local_start_image - b["clip_vision_output"] = local_clip_vision_output - print("🌐 [XB-BOX] 总线状态已更新：独立参考图已成为下游节点的新全局空间基准。") - - _refresh_models() - return (b, accumulated_video, accumulated_audio) - - -# ============================================================ -# XB_WanInfiniteTalk_RelayNode_AllInOne — InfiniteTalk 全能接力点 -# ============================================================ -class XB_WanInfiniteTalk_RelayNode_AllInOne: - """ - InfiniteTalk 无限对口型接力点（全能版）。 - 首帧来源三选一：独立参考图 | 继承总线全局图 | 继承前段尾帧。 - 自动对齐音频裁剪与重叠帧，确保跳帧和长度一致。 - """ - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] - files = sorted(files) - if not files: - files = ["[Folder empty_Please connect or upload]"] - - return { - "required": { - "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), - "positive_prompt": ("STRING", {"multiline": True, "default": ""}), - "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "start_image_mode": (["继承总线全局图", "独立参考图", "继承前段尾帧"], {"default": "继承总线全局图"}), - "ref_image_file": (files, {"image_upload": True}), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）。音频和视频裁剪会自动对齐"}), - "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), - "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), - }, - "optional": { - "prev_video": ("IMAGE",), - "prev_audio": ("AUDIO",), - "audio": ("AUDIO",), - } - } - - RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") - RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") - FUNCTION = "execute_relay" - CATEGORY = "XB_ToolBox/Pipeline" - - def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, start_image_mode="继承总线全局图", ref_image_file="", - motion_frame_count=9, relay_count=1, total_frames_display="", - prev_video=None, prev_audio=None, audio=None): - b = wan_infinitetalk_bus.copy() - fps = b.get("fps", 25.0) - bus_audio_mode = b.get("_bus_audio_mode", False) - print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk AllInOne Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 模式: {start_image_mode}") - - accumulated_video = prev_video - accumulated_audio = prev_audio - - # --- 预处理独立参考图 --- - is_local_ref = (start_image_mode == "独立参考图") - is_tail_frame = (start_image_mode == "继承前段尾帧") - local_start_image = None - local_clip_vision_output = None - if is_local_ref: - if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": - raise ValueError("🚨 [XB-BOX] 全能接力点已开启独立参考图，但未选择图片文件！") - image_path = folder_paths.get_annotated_filepath(ref_image_file) - if not os.path.exists(image_path): - raise ValueError(f"🚨 [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}") - i = Image.open(image_path) - i = ImageOps.exif_transpose(i) - image_data = i.convert("RGB") - image_data = np.array(image_data).astype(np.float32) / 255.0 - local_start_image = torch.from_numpy(image_data)[None,] - cv_model = b.get("clip_vision") - if cv_model is None: - raise ValueError("🚨 [XB-BOX] 独立参考图需要总线连接 clip_vision 模型！") - w, h = b["width"], b["height"] - sm = b.get("scale_method", "lanczos") - cm = b.get("crop_mode", "center") - clip_img = comfy.utils.common_upscale(local_start_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) - local_clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") - print(f"🖼️ [XB-BOX] 全能接力点使用独立参考图 ({image_data.shape[1]}×{image_data.shape[0]} → {w}×{h}, {sm}/{cm})") - - completed = 0 - for r in range(relay_count): - print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") - - if bus_audio_mode: - total_frames = b["_total_frames"] - accumulated = b["_accumulated_frames"] - remaining = total_frames - accumulated - - if remaining <= 0: - break - - if remaining < segment_length: - if accumulated == 0: - need_raw = remaining - else: - need_raw = remaining + motion_frame_count - actual_length = ((need_raw + 2) // 4) * 4 + 1 - if actual_length < 1: - break - print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") - else: - actual_length = segment_length - - encoded_audio = b["_encoded_audio"] - raw_audio = b.get("_raw_audio") - use_segment_audio = False - print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") - else: - if audio is None: - print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") - break - - actual_length = segment_length - encoded_audio = None - raw_audio = audio - use_segment_audio = True - print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") - - # --- 帧数对齐 4N+1 --- - raw_length = actual_length - aligned_length = ((actual_length + 2) // 4) * 4 + 1 - if aligned_length != raw_length: - print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") - if not bus_audio_mode and audio is not None: - pad_frames = aligned_length - raw_length - pad_sec = pad_frames / fps - sr = audio["sample_rate"] - pad_samples = int(pad_sec * sr) - pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) - raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} - actual_length = aligned_length - - if not bus_audio_mode: - ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") - if ae_cls is None: - raise ImportError("AudioEncoderEncode not found") - ae = ae_cls() - encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) - - pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) - neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) - - prev = b.get("_previous_frames") - global_frame_offset = b["_accumulated_frames"] if bus_audio_mode else None - - # --- 动态决定当前段的首帧来源 --- - # 🔑 r==0 尊重用户选择; r>0 强制切换为"继承前段尾帧"，防止姿势回拉 - current_step_mode = start_image_mode if r == 0 else "继承前段尾帧" - - if current_step_mode == "独立参考图": - cur_start_image = local_start_image - cur_clip_output = local_clip_vision_output - # 💥 Hard Cut: 换参考图=镜头切换，必须清空前置重叠帧 - # 否则旧画面的运动残影会强行揉进新画面，导致 20~40 帧的"恐怖渐变" - prev = None - print("🎬 [XB-BOX] 触发镜头硬切 (Hard Cut): 已清空前置重叠帧，启用独立参考图") - - elif current_step_mode == "继承前段尾帧": - if prev is not None: - is4d_prev = len(prev.shape) == 4 - cur_start_image = prev[-1:] if is4d_prev else prev[:, -1:] - - # VAE 解码可能输出 5D，必须压缩为 4D - if not is4d_prev: - while len(cur_start_image.shape) > 4: - cur_start_image = cur_start_image[:, 0] - - cv_model = b.get("clip_vision") - if cv_model is not None: - w, h = b["width"], b["height"] - sm = b.get("scale_method", "lanczos") - cm = b.get("crop_mode", "center") - clip_input = cur_start_image.clamp(0.0, 1.0).movedim(-1, 1) - clip_img = comfy.utils.common_upscale(clip_input, w, h, sm, cm).movedim(1, -1) - cur_clip_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") - print(f"🔗 [XB-BOX] 尾帧推演: prev={prev.shape}→start={cur_start_image.shape}, clip_vision 已重新编码") - else: - cur_clip_output = b.get("clip_vision_output") - print(f"🔗 [XB-BOX] 尾帧推演: prev={prev.shape}→start={cur_start_image.shape}, 无 clip_vision") - else: - cur_start_image = b.get("start_image") - cur_clip_output = b.get("clip_vision_output") - print("⚠️ [XB-BOX] 尾帧推演无前置帧，回退到总线全局图") - - else: # "继承总线全局图" - cur_start_image = b.get("start_image") - cur_clip_output = b.get("clip_vision_output") - print("🌐 [XB-BOX] 全局图模式: 继承总线原始设定") - core = XB_WanInfiniteTalkToVideo_Single() - model_out, pos, neg, latent, trim = core.process( - model=b["model"], model_patch=b["model_patch"], - positive=pos_cond, negative=neg_cond, vae=b["vae"], - width=b["width"], height=b["height"], length=actual_length, - audio_encoder_output_1=encoded_audio, - motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], - vae_tile_size=b["vae_encode_tile_size"], - clip_vision_output=cur_clip_output, - start_image=cur_start_image, previous_frames=prev, - segment_audio=use_segment_audio, - global_frame_offset=global_frame_offset, - scale_method=b.get("scale_method", "lanczos"), - crop_mode=b.get("crop_mode", "center"), - ) - - latent_out, = nodes.KSampler().sample( - model=model_out, seed=b["seed"], - steps=b["steps"], cfg=b["cfg"], - sampler_name=b["sampler_name"], scheduler=b["scheduler"], - positive=pos, negative=neg, latent_image=latent, denoise=1.0, - ) - - decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) - - is4d = len(decoded.shape) == 4 - cd = 0 if is4d else 1 - total = decoded.shape[cd] - b["_previous_frames"] = decoded - - # 🛡️ 关键对齐：trim_val 来自模型返回的重叠帧数，音频和视频裁剪统一使用此值 - trim_val = int(trim) if trim else 0 - trim_val = min(trim_val, total - 1) if trim_val < total else 0 - cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] - final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkAIO-{r+1}", concat_mode=b.get("concat_mode", "自动")) - - # 🛡️ 音频裁剪与视频裁剪严格对齐：使用同一个 trim_val - ats = trim_val / fps if trim_val else 0.0 - if bus_audio_mode: - raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) - raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 - # 对齐到 trim_val 帧：从当前累计位置开始，取 actual_out 帧长度的音频 - actual_out_frames = cut.shape[cd] - seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) - seg_end_sample = int((b["_accumulated_frames"] + actual_out_frames + trim_val) / fps * raw_sr) - seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) - cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None - else: - cut_audio = raw_audio - if ats > 0 and raw_audio: - wf = raw_audio["waveform"] - sr = raw_audio["sample_rate"] - n = int(ats * sr) - if n < wf.shape[-1]: - cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} - else: - cut_audio = None - - if accumulated_audio is None: - final_audio = cut_audio - elif cut_audio is None: - final_audio = accumulated_audio - else: - pa, ca = accumulated_audio, cut_audio - if pa["sample_rate"] != ca["sample_rate"]: - if torchaudio is None: - raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") - ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) - ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} - pa_wf = pa["waveform"] - ca_wf = ca["waveform"].to(pa_wf.device) - if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: - if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: - rpt = [1] * pa_wf.dim(); rpt[-2] = ca_wf.shape[-2]; pa_wf = pa_wf.repeat(*rpt) - elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: - rpt = [1] * ca_wf.dim(); rpt[-2] = pa_wf.shape[-2]; ca_wf = ca_wf.repeat(*rpt) - final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} - - b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] - if bus_audio_mode: - b["_accumulated_frames"] += cut.shape[cd] - accumulated_video = final_video - accumulated_audio = final_audio - print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f (trim={trim_val}f) total={final_video.shape[cd]}f") - completed = r + 1 - - if completed < relay_count: - print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") - else: - print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") - # 👑 刷新总线全局状态：如果引入了独立参考图，将其升级为下游节点的新基准 - if is_local_ref: - b["start_image"] = local_start_image - b["clip_vision_output"] = local_clip_vision_output - print("🌐 [XB-BOX] 总线状态已更新：独立参考图已成为下游节点的新全局空间基准。") - - _refresh_models() - return (b, accumulated_video, accumulated_audio) +import os +import torch +import numpy as np +import gc +from PIL import Image, ImageOps +import nodes +import comfy.samplers +import comfy.model_management as mm +import folder_paths +try: + import torchaudio +except ImportError: + torchaudio = None + print("⚠️ [XB-BOX] torchaudio 未安装！音频重采样功能将不可用。请运行: pip install torchaudio") +from .nodes_wan_vae import XB_WanFirstLastFrameToVideo, XB_WanImageToVideo, XB_WanInfiniteTalkToVideo_Single + + +# VRAM 保护阈值：累积视频超过总显存此比例时自动迁移至 CPU（兼容 6GB~48GB 全系列显卡） +_VRAM_VIDEO_SAFETY_RATIO = 0.30 + + +def _get_vram_info(): + """获取当前 CUDA 设备显存信息 (total, free, used)""" + if not torch.cuda.is_available(): + return 0, 0, 0 + d = torch.cuda.current_device() + total = torch.cuda.get_device_properties(d).total_memory + reserved = torch.cuda.memory_reserved(d) + allocated = torch.cuda.memory_allocated(d) + free = total - reserved + return total, free, allocated + + +def _refresh_models(force=False): + """VRAM 感知缓存刷新：仅在显存紧张时清理，避免频繁清空分配池导致碎片化""" + if not force: + _, free, _ = _get_vram_info() + if torch.cuda.is_available(): + total = torch.cuda.get_device_properties(torch.cuda.current_device()).total_memory + if free > total * 0.12: + return + if torch.cuda.is_available(): + torch.cuda.synchronize() + torch.cuda.empty_cache() + elif torch.backends.mps.is_available(): + torch.mps.synchronize() + torch.mps.empty_cache() + gc.collect() + + +def _safe_video_accumulate(prev_video, new_segment, concat_dim, label="", concat_mode="自动"): + """ + VRAM 感知视频累积拼接，兼容从 6GB 到 48GB 的所有配置。 + concat_mode: "自动"(默认) | "强制GPU" | "强制CPU" + """ + if prev_video is None: + return new_segment + + target_device = new_segment.device + + # 强制CPU + if concat_mode == "强制CPU": + new_segment = new_segment.cpu() + if prev_video.device.type != 'cpu': + prev_video = prev_video.cpu() + result = torch.cat([prev_video, new_segment], dim=concat_dim) + tag = f"[{label}] " if label else "" + print(f"💾 [XB-BOX] {tag}强制CPU拼接") + return result + + # 强制GPU + if concat_mode == "强制GPU": + if prev_video.device != target_device: + prev_video = prev_video.to(target_device, non_blocking=True) + result = torch.cat([prev_video, new_segment], dim=concat_dim) + tag = f"[{label}] " if label else "" + print(f"🚀 [XB-BOX] {tag}强制GPU拼接: {result.numel()*result.element_size()/1024**3:.2f}GB") + return result + + # 自动模式 + combined_bytes = (prev_video.numel() + new_segment.numel()) * new_segment.element_size() + if target_device.type == 'cuda': + total_mem = torch.cuda.get_device_properties(target_device).total_memory + ratio = combined_bytes / total_mem + else: + ratio = 0.0 + + if prev_video.device.type == 'cpu' or ratio > _VRAM_VIDEO_SAFETY_RATIO: + new_segment = new_segment.cpu() + if prev_video.device.type != 'cpu': + prev_video = prev_video.cpu() + result = torch.cat([prev_video, new_segment], dim=concat_dim) + tag = f"[{label}] " if label else "" + print(f"💾 [XB-BOX] {tag}VRAM保护: 累积视频 {combined_bytes/1024**3:.2f}GB ({ratio*100:.0f}% 总显存) → 已在 CPU 完成无损拼接") + return result + + if prev_video.device != target_device: + prev_video = prev_video.to(target_device, non_blocking=True) + return torch.cat([prev_video, new_segment], dim=concat_dim) + + +def _match_color_to_ref(target_img, ref_img): + """色彩重映射引擎：用均值和标准差将 target 光影强制对齐到 ref。 + + 3D VAE 的边界帧因缺失未来帧 padding 会变暗——此函数用数学手段 + 把变暗的尾帧拉回原始首帧的光影分布，斩断接力中的曝光衰减死循环。 + + target_img: [1, H, W, 3] (变暗的尾帧) + ref_img: [1, H', W', 3] (总线里的全局参考首帧) + """ + t = target_img.movedim(-1, 1) # [1, 3, H, W] + r = ref_img.movedim(-1, 1) # [1, 3, H', W'] + + t_mean = t.mean(dim=(2, 3), keepdim=True) + t_std = t.std(dim=(2, 3), keepdim=True) + 1e-6 + r_mean = r.mean(dim=(2, 3), keepdim=True) + r_std = r.std(dim=(2, 3), keepdim=True) + 1e-6 + + matched = (t - t_mean) / t_std * r_std + r_mean + return torch.clamp(matched, 0.0, 1.0).movedim(1, -1) + + +def _apply_progressive_color_correction(decoded_image, ref_image, is_4d): + """全序列渐变色彩补偿：平方权重 + 50% 混合降敏，避免尾端色彩过度补偿伪影。 + + 用 start_image 做色彩锚点。平方权重 (x²) 比三次幂 (x³) 过渡更平缓， + 在 33 帧场景下后 15% 帧的修正强度从 73% 降至 60%。 + 50% blending (原帧×0.5 + 匹配帧×0.5) 进一步抑制 color transfer 的内容差异伪影。 + """ + vid = decoded_image if is_4d else decoded_image[0] + T = vid.shape[0] + if T <= 1: + return decoded_image + + raw_last = vid[-1:] # [1, H, W, C] + full_match = _match_color_to_ref(raw_last, ref_image) + + # 🌟 50% 混合降敏：一半原帧 + 一半色彩匹配，抑制内容差异导致的过度补偿 + blended_last = raw_last * 0.5 + full_match * 0.5 + + drift_diff = blended_last - raw_last + linear = torch.linspace(0.0, 1.0, T, device=vid.device, dtype=vid.dtype) + weights = (linear ** 2).view(T, 1, 1, 1) # 平方权重，更平缓 + + corrected_vid = vid + drift_diff * weights + corrected_vid = torch.clamp(corrected_vid, 0.0, 1.0) + + return corrected_vid if is_4d else corrected_vid.unsqueeze(0) + + +# ============================================================ +# XB_Wan_ParamBus — Wan 视频参数总线 +# ============================================================ +class XB_Wan_ParamBus: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model_high": ("MODEL",), + "model_low": ("MODEL",), + "clip": ("CLIP",), + "vae": ("VAE",), + "negative_prompt": ("STRING", {"multiline": True, "default": "vivid colors, overexposed, static, blurry details..."}), + "width": ("INT", {"forceInput": True}), + "height": ("INT", {"forceInput": True}), + "length": ("INT", {"forceInput": True}), + "fps": ("FLOAT", {"forceInput": True}), + # --- VAE 时空分块与重叠参数 --- + "vae_encode_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + "vae_decode_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), + "temporal_chunk_size": ("INT", {"default": 64, "min": 0, "max": 8192, "step": 4}), + "temporal_overlap": ("INT", {"default": 8, "min": 0, "max": 8192, "step": 4}), + "steps": ("INT", {"default": 4, "min": 1, "max": 100}), + "high_noise_steps": ("INT", {"default": 2, "min": 1, "max": 50}), + "cfg": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.1}), + "sampler_name": (comfy.samplers.KSAMPLER_NAMES, ), + "scheduler": (comfy.samplers.SCHEDULER_NAMES, ), + "seed": ("INT", {"default": 123456789}), + "cut_first_frame": ("BOOLEAN", {"default": True, "label_on": "Yes (Relay Deduplication)", "label_off": "No (Keep First Frame)"}), + "trim_head_frames": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), + }, + "optional": { + "clip_vision": ("CLIP_VISION",), + "global_start_image": ("IMAGE",), + "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), + "scale_method": (["lanczos", "bilinear", "bicubic", "nearest-exact", "area"], {"default": "lanczos"}), + "crop_mode": (["center", "disabled"], {"default": "center"}), + } + } + + RETURN_TYPES = ("WAN_BUS",) + RETURN_NAMES = ("📦 WAN_BUS (Dynamic Bus)",) + FUNCTION = "pack_bus" + CATEGORY = "XB_ToolBox/Pipeline" + + def pack_bus(self, **kwargs): + import comfy.utils + cv = kwargs.get("clip_vision") + ref = kwargs.get("global_start_image") # 首尾帧/图生视频用 global_start_image 做视觉参考 + if cv is not None and ref is not None: + w, h = kwargs["width"], kwargs["height"] + method = kwargs.get("scale_method", "lanczos") + crop = kwargs.get("crop_mode", "center") + scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) + kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") + print(f"👁️ [XB-BOX] CLIP视觉编码 ({ref.shape[1]}×{ref.shape[2]} → {w}×{h}, {method}/{crop})") + else: + kwargs["clip_vision_output"] = None + return (kwargs,) + + +# ============================================================ +# XB_Wan_RelayNode — Wan 首尾帧接力点 +# ============================================================ +class XB_Wan_RelayNode: + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] + files = sorted(files) + + if not files: + files = ["[Folder empty_Please connect or upload]"] + + return { + "required": { + "wan_bus": ("WAN_BUS",), + "start_image": ("IMAGE",), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), + "end_image_file": (files, {"image_upload": True}) + }, + "optional": { + "prev_video": ("IMAGE",), + "opt_end_image": ("IMAGE",) + } + } + + @classmethod + def VALIDATE_INPUTS(cls, **kwargs): + return True + + RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") + RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_bus, start_image, positive_prompt, end_image_file, opt_end_image=None, prev_video=None): + print("\n🏃 [XB-BOX] Executing First/Last frame constraint relay task...") + b = wan_bus + seed = b["seed"] + cut_first_frame = b.get("cut_first_frame", True) + trim_head_frames = b.get("trim_head_frames", 1) + + encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) + decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) + spat_overlap = b.get("spatial_overlap", 32) + temp_chunk = b.get("temporal_chunk_size", 64) + temp_overlap = b.get("temporal_overlap", 8) + + if opt_end_image is not None: + end_image = opt_end_image + print(f"🖼️ [XB-BOX] Endpoint connection detected, prioritizing opt_end_image payload.") + else: + if end_image_file == "[Folder empty_Please connect or upload]": + print("⏭️ [XB-BOX] No end frame specified for this segment, skipping relay iteration.") + return (wan_bus, start_image, prev_video) + + image_path = folder_paths.get_annotated_filepath(end_image_file) + if not os.path.exists(image_path): + print(f"⏭️ [XB-BOX] End frame file not found: {os.path.basename(image_path)}, skipping relay iteration.") + return (wan_bus, start_image, prev_video) + + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + image_data = i.convert("RGB") + image_data = np.array(image_data).astype(np.float32) / 255.0 + end_image = torch.from_numpy(image_data)[None,] + + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + cv_output = b.get("clip_vision_output") + pos, neg, latent = XB_WanFirstLastFrameToVideo().process( + positive=pos_cond, negative=neg_cond, vae=b["vae"], + clip_vision_start_image=cv_output, clip_vision_end_image=None, + start_image=start_image, end_image=end_image, + width=b["width"], height=b["height"], length=b["length"], + batch_size=1, vae_tile_size=encode_tile, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + # 🛡️ 护城河：步数边界校验，防止 ODE 时间步崩塌 + total_steps = b["steps"] + high_steps = b["high_noise_steps"] + if high_steps >= total_steps: + high_steps = total_steps - 1 + print(f"⚠️ [XB-BOX] 拦截到致命参数：强制修正 high_noise_steps 为 {high_steps}") + + print(f"🔥 [XB-BOX] Samping Layer 1 (0 -> {high_steps} steps)...") + latent_high, = nodes.KSamplerAdvanced().sample( + model=b["model_high"], add_noise="enable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, + start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" + ) + + print(f"❄️ [XB-BOX] Samping Layer 2 ({high_steps} -> {total_steps})...") + latent_low, = nodes.KSamplerAdvanced().sample( + model=b["model_low"], add_noise="disable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent_high, + start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" + ) + + decoded_image, = nodes.VAEDecode().decode( + samples=latent_low, + vae=b["vae"], + ) + + is_4d = len(decoded_image.shape) == 4 + frame_dim = 0 if is_4d else 1 + total_frames = decoded_image.shape[frame_dim] + + if cut_first_frame and total_frames > 1: + safe_trim = min(trim_head_frames, total_frames - 1) + if safe_trim > 0: + if is_4d: + decoded_image = decoded_image[safe_trim:, :, :, :] + else: + decoded_image = decoded_image[:, safe_trim:, :, :, :] + + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + + # 👑 全序列渐变补偿：start_image 做锚点（接力中=上一段修正尾帧，同色彩空间） + decoded_image = _apply_progressive_color_correction(decoded_image, start_image, is_4d) + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + print("✨ [XB-BOX] 触发抗衰减机制：已应用全序列 3D VAE 曝光漂移渐变补偿！") + + concat_dim = 0 if is_4d else 1 + + final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="Relay") + + _refresh_models() + return (wan_bus, last_frame, final_video) + + +# ============================================================ +# XB_Wan_InfiniteRelayNode — Wan 无限长接力点 +# ============================================================ +class XB_Wan_InfiniteRelayNode: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "wan_bus": ("WAN_BUS",), + "start_image": ("IMAGE",), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), + }, + "optional": { + "prev_video": ("IMAGE",) + } + } + + @classmethod + def VALIDATE_INPUTS(cls, **kwargs): + return True + + RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") + RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_bus, start_image, positive_prompt, prev_video=None): + print("\n🏃‍♀️ [XB-BOX] Executing Image-to-Video Infinite pipeline generation task...") + b = wan_bus + seed = b["seed"] + cut_first_frame = b.get("cut_first_frame", True) + trim_head_frames = b.get("trim_head_frames", 1) + + encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) + decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) + spat_overlap = b.get("spatial_overlap", 32) + temp_chunk = b.get("temporal_chunk_size", 64) + temp_overlap = b.get("temporal_overlap", 8) + + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + cv_output = b.get("clip_vision_output") + pos, neg, latent = XB_WanImageToVideo().process( + positive=pos_cond, negative=neg_cond, vae=b["vae"], + clip_vision_output=cv_output, start_image=start_image, + width=b["width"], height=b["height"], length=b["length"], + batch_size=1, vae_tile_size=encode_tile, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + # 🛡️ 护城河：步数边界校验，防止 ODE 时间步崩塌 + total_steps = b["steps"] + high_steps = b["high_noise_steps"] + if high_steps >= total_steps: + high_steps = total_steps - 1 + print(f"⚠️ [XB-BOX] 拦截到致命参数：强制修正 high_noise_steps 为 {high_steps}") + + print(f"🔥 [XB-BOX] Samping Layer 1 (0 -> {high_steps} steps)...") + latent_high, = nodes.KSamplerAdvanced().sample( + model=b["model_high"], add_noise="enable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, + start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" + ) + + print(f"❄️ [XB-BOX] Samping Layer 2 ({high_steps} -> {total_steps})...") + latent_low, = nodes.KSamplerAdvanced().sample( + model=b["model_low"], add_noise="disable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent_high, + start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" + ) + + decoded_image, = nodes.VAEDecode().decode( + samples=latent_low, + vae=b["vae"], + ) + + is_4d = len(decoded_image.shape) == 4 + frame_dim = 0 if is_4d else 1 + total_frames = decoded_image.shape[frame_dim] + + if cut_first_frame and total_frames > 1: + safe_trim = min(trim_head_frames, total_frames - 1) + if safe_trim > 0: + if is_4d: + decoded_image = decoded_image[safe_trim:, :, :, :] + else: + decoded_image = decoded_image[:, safe_trim:, :, :, :] + + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + + # 👑 全序列渐变补偿：start_image 做锚点（接力中=上一段修正尾帧，同色彩空间） + decoded_image = _apply_progressive_color_correction(decoded_image, start_image, is_4d) + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + print("✨ [XB-BOX] 触发抗衰减机制：已应用全序列 3D VAE 曝光漂移渐变补偿！") + + concat_dim = 0 if is_4d else 1 + + final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="InfiniteRelay") + + _refresh_models() + return (wan_bus, last_frame, final_video) + + +# ============================================================ +# XB_Video_Merger — 视频拼接器 +# ============================================================ +class XB_Video_Merger: + @classmethod + def INPUT_TYPES(cls): + inputs = {"required": {}, "optional": {}} + for i in range(1, 11): + inputs["optional"][f"video_{i}"] = ("IMAGE",) + return inputs + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("🎞️ Full Long Video",) + FUNCTION = "merge_videos" + CATEGORY = "XB_ToolBox/Pipeline" + + def merge_videos(self, **kwargs): + videos = [kwargs.get(f"video_{i}") for i in range(1, 11) if kwargs.get(f"video_{i}") is not None] + if not videos: + raise ValueError("🚨 [XB-BOX] Merge failed: At least one video must be connected!") + + concat_dim = 0 if len(videos[0].shape) == 4 else 1 + ref_shape = list(videos[0].shape) + for i, vid in enumerate(videos): + for d in range(len(ref_shape)): + if d != concat_dim and vid.shape[d] != ref_shape[d]: + raise ValueError( + f"🚨 [XB-BOX] Merge failed: Video segment {i+1} shape {list(vid.shape)} " + f"does not match first segment shape {ref_shape} on dimension {d} " + f"(expected {ref_shape[d]}, got {vid.shape[d]})" + ) + # 将全部视频段卸载到 CPU 再拼接，防止多段同时驻留 VRAM 导致 OOM + videos_cpu = [v.cpu() for v in videos] + merged = torch.cat(videos_cpu, dim=concat_dim) + print(f"💾 [XB-BOX] Merger: {len(videos)} 段视频已在 CPU 完成拼接 (shape={list(merged.shape)})") + return (merged,) + +# ============================================================ +# XB_StoryboardSlicer — 分镜切片器 +# ============================================================ +class XB_StoryboardSlicer: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "image": ("IMAGE",), + "mode": (["3-Grid (1x3 Horizontal)", "3-Grid (3x1 Vertical)", "4-Grid (2x2)", "6-Grid (2x3 Horizontal)", "6-Grid (3x2 Vertical)", "9-Grid (3x3)"], {"default": "4-Grid (2x2)"}), + "crop_margin": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 0.45, "step": 0.01}), + } + } + RETURN_TYPES = ("IMAGE",) * 9 + RETURN_NAMES = ("Img1", "Img2", "Img3", "Img4", "Img5", "Img6", "Img7", "Img8", "Img9") + FUNCTION = "slice_image" + CATEGORY = "XB_ToolBox/Pipeline" + + def slice_image(self, image, mode, crop_margin): + if "1x3" in mode: rows, cols = 1, 3 + elif "3x1" in mode: rows, cols = 3, 1 + elif "2x2" in mode: rows, cols = 2, 2 + elif "2x3" in mode: rows, cols = 2, 3 + elif "3x2" in mode: rows, cols = 3, 2 + elif "3x3" in mode: rows, cols = 3, 3 + else: rows, cols = 2, 2 + + batch_size, h, w, channels = image.shape + slice_h, slice_w = h // rows, w // cols + crop_y, crop_x = int(slice_h * crop_margin), int(slice_w * crop_margin) + + sliced_images = [] + for r in range(rows): + for c in range(cols): + sy, ey = r * slice_h + crop_y, (r + 1) * slice_h - crop_y + sx, ex = c * slice_w + crop_x, (c + 1) * slice_w - crop_x + if sy >= ey: ey = sy + 1 + if sx >= ex: ex = sx + 1 + sliced_images.append(image[:, sy:ey, sx:ex, :]) + + placeholder_h = max(1, slice_h - crop_y * 2) + placeholder_w = max(1, slice_w - crop_x * 2) + while len(sliced_images) < 9: + sliced_images.append(torch.zeros((1, placeholder_h, placeholder_w, 3), dtype=torch.float32, device=image.device)) + return tuple(sliced_images) + + +# ============================================================ +# XB_WanAnimate_ParamBus — Wan Animate 动作迁移参数总线 +# ============================================================ +class XB_WanAnimate_ParamBus: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": ("MODEL",), + "clip": ("CLIP",), + "vae": ("VAE",), + "negative_prompt": ("STRING", {"multiline": True, "default": "vivid colors, overexposed, static, blurry details..."}), + "width": ("INT", {"default": 480, "min": 260, "step": 16}), + "height": ("INT", {"default": 832, "step": 16}), + "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "0=自动从源视频取长度；>0=强制截断到此帧数"}), + "fps": ("FLOAT", {"default": 16.0}), + # --- VAE 时空分块与重叠参数 --- + "vae_encode_tile_size": ("INT", {"default": 320, "step": 32}), + "vae_decode_tile_size": ("INT", {"default": 320, "step": 32}), + "spatial_overlap": ("INT", {"default": 32, "step": 32}), + "temporal_chunk_size": ("INT", {"default": 64, "step": 4}), + "temporal_overlap": ("INT", {"default": 8, "step": 4}), + # --- 采样与控制 --- + "steps": ("INT", {"default": 20, "min": 1, "max": 100}), + "cfg": ("FLOAT", {"default": 5.0, "step": 0.1}), + "sampler_name": (comfy.samplers.KSAMPLER_NAMES, ), + "scheduler": (comfy.samplers.SCHEDULER_NAMES, ), + "seed": ("INT", {"default": 123456789}), + "continue_motion_max_frames": ("INT", {"default": 5, "min": 0, "max": 16}), + # --- 显存清理 --- + "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), + # --- CLIP视觉 --- + "clip_vision": ("CLIP_VISION",), + }, + "optional": { + "global_ref_image": ("IMAGE",), + "pose_video": ("IMAGE",), + "face_video": ("IMAGE",), + "background_video": ("IMAGE",), + "character_mask": ("MASK",), + "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), + "scale_method": (["lanczos", "bilinear", "bicubic", "nearest-exact", "area"], {"default": "lanczos"}), + "crop_mode": (["center", "disabled"], {"default": "center"}), + } + } + + RETURN_TYPES = ("WAN_ANIMATE_BUS",) + RETURN_NAMES = ("📦 WAN_ANIMATE_BUS",) + FUNCTION = "pack_bus" + CATEGORY = "XB_ToolBox/Pipeline" + + def pack_bus(self, **kwargs): + kwargs["current_offset"] = 0 + cv = kwargs.get("clip_vision") + ref = kwargs.get("global_ref_image") + if cv is not None and ref is not None: + w, h = kwargs.get("width", 480), kwargs.get("height", 832) + m = kwargs.get("scale_method", "lanczos") + c = kwargs.get("crop_mode", "center") + scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, m, c).movedim(1, -1) + kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") + print(f"👁️ [XB-BOX] Animate CLIP视觉编码 ({ref.shape[2]}×{ref.shape[1]} → {w}×{h}, {m}/{c})") + return (kwargs,) + + +# ============================================================ +# XB_WanAnimate_RelayNode — Wan Animate 无限接力点 +# ============================================================ +class XB_WanAnimate_RelayNode: + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] + files = sorted(files) + + if not files: + files = ["[Folder empty_Please connect or upload]"] + + return { + "required": { + "wan_animate_bus": ("WAN_ANIMATE_BUS",), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), + "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), + "ref_image_file": (files, {"image_upload": True}), + }, + "optional": { + "prev_video": ("IMAGE",), + } + } + + @classmethod + def VALIDATE_INPUTS(cls, **kwargs): + return True + + RETURN_TYPES = ("WAN_ANIMATE_BUS", "IMAGE") + RETURN_NAMES = ("📦 WAN_ANIMATE_BUS (传给下段)", "🎞️ 累加视频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_animate_bus, segment_length, positive_prompt, use_local_ref_image, ref_image_file, prev_video=None, opt_ref_image=None): + b = wan_animate_bus.copy() + current_offset = b.get("current_offset", 0) + seed = b.get("seed", 123456789) + + source_video = b.get("pose_video") if b.get("pose_video") is not None else b.get("face_video") + total_source_frames = float('inf') + + # 优先使用总线 total_frames，其次用源视频长度 + bus_total = b.get("total_frames", 0) + if bus_total > 0: + total_source_frames = bus_total + elif source_video is not None: + total_source_frames = source_video.shape[0] if len(source_video.shape) == 4 else source_video.shape[1] + + remaining_frames = max(0, total_source_frames - current_offset) + cont_max = b.get("continue_motion_max_frames", 5) + + if remaining_frames <= 0: + print(f"\n⏭️ [XB-BOX] 动作视频已耗尽 (已跑到 {current_offset} 帧 / 总 {total_source_frames} 帧)。当前接力点直接跳过！") + fallback_video = prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3)) + return (b, fallback_video) + + if remaining_frames < segment_length: + # 最后一段：trim 会吃掉 continue_motion 帧，需补回；再对齐 4N+1 + if current_offset == 0: + need_raw = remaining_frames # 第一段无 trim + else: + need_raw = remaining_frames + cont_max + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + print(f"⏭️ [XB-BOX] 剩余 {remaining_frames} 帧不足以生成最小段，跳过") + fallback_video = prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3)) + return (b, fallback_video) + print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining_frames} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + print(f"\n🏃‍♀️ [XB-BOX] Executing Wan Animate Relay task... Target segment length: {actual_length}") + + is_local = (use_local_ref_image == "独立参考图" or + use_local_ref_image is True or + use_local_ref_image == 1) + if is_local: + if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": + print("⏭️ [XB-BOX] 已开启独立参考图但未选择图片文件，跳过当前 Animate 接力点。") + return (b, prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3))) + image_path = folder_paths.get_annotated_filepath(ref_image_file) + if not os.path.exists(image_path): + print(f"⏭️ [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}，跳过当前 Animate 接力点。") + return (b, prev_video if prev_video is not None else torch.zeros((1, b.get("height", 832), b.get("width", 480), 3))) + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + image_data = i.convert("RGB") + image_data = np.array(image_data).astype(np.float32) / 255.0 + ref_image = torch.from_numpy(image_data)[None,] + cv_model = b.get("clip_vision") + if cv_model is None: + raise ValueError("🚨 [XB-BOX] 使用独立参考图需要总线连接 clip_vision 模型！") + # CLIP 编码用总线缩放参数，不做预缩放（XB_WanAnimateToVideo 内部自己处理 VAE 编码） + w, h = b.get("width", 480), b.get("height", 832) + sm = b.get("scale_method", "lanczos") + cm = b.get("crop_mode", "center") + clip_img = comfy.utils.common_upscale(ref_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) + clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") + print("🖼️ [XB-BOX] 使用独立参考图（含独立CLIP视觉编码）。") + else: + ref_image = b.get("global_ref_image") + if ref_image is None: + raise ValueError("🚨 [XB-BOX] 缺少参考图！请开启独立参考图选择图片，或在总线上连接 global_ref_image。") + clip_vision_output = b.get("clip_vision_output") + print("🌐 [XB-BOX] 使用总线全局参考图。") + + prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) + + continue_motion = None + cont_max_frames = b.get("continue_motion_max_frames", 5) + if is_local: + cont_max_frames = 0 # 独立参考图时关闭衔接帧，避免CLIP视觉特征冲突 + if prev_video is not None and cont_max_frames > 0: + is_4d = len(prev_video.shape) == 4 + if is_4d: + continue_motion = prev_video[-cont_max_frames:, :, :, :] + else: + continue_motion = prev_video[:, -cont_max_frames:, :, :, :] + + try: + from .nodes_wan import XB_WanAnimateToVideo + except ImportError: + raise ImportError("🚨 [XB-BOX] 找不到 XB_WanAnimateToVideo 底座节点，请检查 nodes_wan.py 文件目录！") + + animate_core = XB_WanAnimateToVideo() + func_name = getattr(XB_WanAnimateToVideo, "FUNCTION", "process") + func = getattr(animate_core, func_name) + + kwargs = { + "positive": pos_cond, + "negative": neg_cond, + "vae": b["vae"], + "clip_vision_output": clip_vision_output, + "reference_image": ref_image, + "face_video": b.get("face_video"), + "pose_video": b.get("pose_video"), + "background_video": b.get("background_video"), + "character_mask": b.get("character_mask"), + "continue_motion": continue_motion, + "width": b["width"], + "height": b["height"], + "length": actual_length, + "batch_size": 1, + "continue_motion_max_frames": cont_max_frames, + "video_frame_offset": current_offset, + "vae_tile_size": b.get("vae_encode_tile_size", 320) + } + + pos, neg, latent, trim_latent, trim_image, new_offset = func(**kwargs) + + print(f"🔥 [XB-BOX] Animate Single-Pass Sampling ({b['steps']} steps)...") + latent_sampled, = nodes.KSampler().sample( + model=b["model"], + seed=seed, + steps=b["steps"], + cfg=b["cfg"], + sampler_name=b["sampler_name"], + scheduler=b["scheduler"], + positive=pos, + negative=neg, + latent_image=latent, + denoise=1.0 + ) + + trim_latent_val = trim_latent if trim_latent is not None else 0 + if trim_latent_val > 0: + latent_sampled["samples"] = latent_sampled["samples"][:, :, trim_latent_val:, :, :] + + try: + from ..core.nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal + except ImportError: + raise ImportError("🚨 [XB-BOX] 找不到 _AliasVAEDecodeTemporal，请确保 nodes_vanilla_wrappers.py 未被删除！") + + cleanup_mode = b.get("cleanup", "不做任何清理") + decoder = XB_ROCmVAEDecodeTemporal() + decoded_image, = decoder.decode( + samples=latent_sampled, + vae=b["vae"], + tile=b.get("vae_decode_tile_size", 320), + overlap=b.get("spatial_overlap", 32), + t_tile=b.get("temporal_chunk_size", 64), + t_overlap=b.get("temporal_overlap", 8), + cleanup=cleanup_mode + ) + + is_4d = len(decoded_image.shape) == 4 + concat_dim = 0 if is_4d else 1 + + # 像素层裁切重叠过渡帧（trim_image 对应 continue_motion_max_frames） + trim_px = int(trim_image) if trim_image else 0 + if trim_px > 0 and trim_px < decoded_image.shape[concat_dim]: + if is_4d: + decoded_image = decoded_image[trim_px:, :, :, :] + else: + decoded_image = decoded_image[:, trim_px:, :, :, :] + print(f"✂️ [XB-BOX] 像素裁切重叠: {trim_px} 帧") + + final_video = _safe_video_accumulate(prev_video, decoded_image, concat_dim, label="AnimateRelay") + + actual_out = decoded_image.shape[concat_dim] + b["current_offset"] = current_offset + actual_out + print(f"✅ [XB-BOX] Animate Relay Complete. trim_lat={trim_latent_val} trim_px={trim_px} out={actual_out}f total={final_video.shape[concat_dim]}f") + + return (b, final_video) + + +# ============================================================ +# XB_WanInfiniteTalk_ParamBus — InfiniteTalk 无限对口型参数总线 +# ============================================================ +class XB_WanInfiniteTalk_ParamBus: + """ + 集中管理所有共享参数。内置音频编码器 + CLIP 视觉编码。 + 接力点只需传正面提示词 + 分段帧数 + 音频，其余全从总线走。 + """ + _SCALE_METHODS = ["bilinear", "bicubic", "lanczos", "nearest-exact", "area"] + _CROP_MODES = ["center", "disabled"] + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + # --- 模型 --- + "model": ("MODEL",), + "model_patch": ("MODEL_PATCH",), + "clip": ("CLIP",), + "vae": ("VAE",), + "audio_encoder": ("AUDIO_ENCODER",), + # --- CLIP视觉 --- + "clip_vision": ("CLIP_VISION",), + # --- 基础参数 --- + "negative_prompt": ("STRING", {"multiline": True, "default": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"}), + "width": ("INT", {"default": 832, "min": 260, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "总线音频模式：音频切片节点输出的总帧数，耗尽后自动停止接力。0=独立音频模式"}), + "fps": ("FLOAT", {"default": 25.0, "min": 1.0, "max": 120.0, "step": 1.0}), + # --- VAE 分块 --- + "vae_encode_tile_size": ("INT", {"default": 256, "min": 0, "max": 3840, "step": 32}), + "vae_decode_tile_size": ("INT", {"default": 192, "min": 0, "max": 3840, "step": 32}), + "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), + "temporal_chunk_size": ("INT", {"default": 96, "min": 0, "max": 8192, "step": 4}), + "temporal_overlap": ("INT", {"default": 4, "min": 0, "max": 8192, "step": 4}), + # --- 采样 --- + "steps": ("INT", {"default": 4, "min": 1, "max": 100}), + "cfg": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.1}), + "sampler_name": (comfy.samplers.KSAMPLER_NAMES,), + "scheduler": (comfy.samplers.SCHEDULER_NAMES,), + "seed": ("INT", {"default": 123456789}), + # --- InfiniteTalk --- + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), + "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + # --- 缩放 --- + "scale_method": (cls._SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (cls._CROP_MODES, {"default": "center"}), + # --- 清理 --- + "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), + }, + "optional": { + "start_image": ("IMAGE",), + "audio": ("AUDIO", {"tooltip": "总线音频模式：接入长音频，总线一次性编码，接力点自动分段"}), + "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置"}), + }, + } + + RETURN_TYPES = ("WAN_INFINITETALK_BUS",) + RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS",) + FUNCTION = "pack_bus" + CATEGORY = "XB_ToolBox/Pipeline" + + def pack_bus(self, **kwargs): + # CLIP 视觉编码：先缩放到视频尺寸，再按 crop 模式处理 + cv = kwargs.get("clip_vision") + ref = kwargs.get("start_image") + if cv is not None and ref is not None: + w, h = kwargs["width"], kwargs["height"] + method = kwargs.get("scale_method", "lanczos") + crop = kwargs.get("crop_mode", "center") + scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) + kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") + print(f"👁️ [XB-BOX] CLIP视觉编码: {ref.shape[1]}×{ref.shape[2]} → {w}×{h} ({method}/{crop})") + else: + kwargs["clip_vision_output"] = None + + # --- 总线音频模式：接入长音频 + 总帧数则一次性编码 --- + audio_input = kwargs.get("audio") + total_frames = kwargs.get("total_frames", 0) + if audio_input is not None and total_frames > 0: + ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") + if ae_cls is None: + raise ImportError("AudioEncoderEncode not found") + ae = ae_cls() + kwargs["_encoded_audio"], = getattr(ae, ae.FUNCTION)(kwargs["audio_encoder"], audio_input) + kwargs["_raw_audio"] = audio_input # 保留原始波形用于输出拼接 + kwargs["_total_frames"] = total_frames + kwargs["_accumulated_frames"] = 0 + kwargs["_bus_audio_mode"] = True + print(f"🔊 [XB-BOX] 总线音频模式：已编码长音频，总目标 {total_frames} 帧 ({total_frames/kwargs['fps']:.1f}s)") + else: + kwargs["_bus_audio_mode"] = False + + kwargs["_previous_frames"] = None + kwargs["_global_frame_offset"] = 0 + return (kwargs,) + + +# ============================================================ +# XB_WanInfiniteTalk_RelayNode — InfiniteTalk 无限对口型接力点 +# ============================================================ +class XB_WanInfiniteTalk_RelayNode: + """ + 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 + 音频独立输入 + 自动裁剪重叠 + 累加输出，与视频帧对齐。 + """ + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), + "positive_prompt": ("STRING", {"multiline": True, "default": ""}), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + }, + "optional": { + "prev_video": ("IMAGE",), + "prev_audio": ("AUDIO",), + "audio": ("AUDIO",), + } + } + + RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") + RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, + prev_video=None, prev_audio=None, audio=None): + b = wan_infinitetalk_bus.copy() + motion = b.get("motion_frame_count", 9) + fps = b.get("fps", 25.0) + bus_audio_mode = b.get("_bus_audio_mode", False) + + # ================================================================ + # 🛡️ 模式判断 & 帧数计算 + # ================================================================ + if bus_audio_mode: + # --- 模式1：总线音频模式 --- + total_frames = b["_total_frames"] + accumulated = b["_accumulated_frames"] + remaining = total_frames - accumulated + + if remaining <= 0: + print(f"\n⏭️ [XB-BOX] 总线帧数已耗尽 ({accumulated}/{total_frames})，跳过接力点") + fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) + fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} + return (b, fb_video, fb_audio) + + if remaining < segment_length: + # 最后一段：trim 会吃掉 motion 帧，需补回；再对齐 4N+1 + if accumulated == 0: + need_raw = remaining # 第一段无 trim + else: + need_raw = remaining + motion + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + print(f"⏭️ [XB-BOX] 剩余 {remaining} 帧不足以生成最小段，跳过") + fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) + fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} + return (b, fb_video, fb_audio) + print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + encoded_audio = b["_encoded_audio"] + raw_audio = b.get("_raw_audio") + use_segment_audio = False # 用 offset 方式裁剪长音频 + print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") + else: + # --- 模式2：独立音频模式 --- + if audio is None: + print("\n⏭️ [XB-BOX] 无音频输入，跳过接力点") + fb_video = prev_video if prev_video is not None else torch.zeros((1, b["height"], b["width"], 3)) + fb_audio = prev_audio if prev_audio is not None else {"waveform": torch.zeros((1, 1)), "sample_rate": 44100} + return (b, fb_video, fb_audio) + + actual_length = segment_length + encoded_audio = None # 下面编码 + raw_audio = audio + use_segment_audio = True # 每段独立音频从头开始 + print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") + + # --- 🛡️ 帧数对齐：Wan 模型要求 length = 4n+1 --- + raw_length = actual_length + aligned_length = ((actual_length + 2) // 4) * 4 + 1 + if aligned_length != raw_length: + print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") + # 总线模式下不补音频（已编码），独立模式下补静音 + if not bus_audio_mode and audio is not None: + pad_frames = aligned_length - raw_length + pad_sec = pad_frames / fps + sr = audio["sample_rate"] + pad_samples = int(pad_sec * sr) + pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) + raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} + actual_length = aligned_length + + # --- 音频编码（独立模式） --- + if not bus_audio_mode: + ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") + if ae_cls is None: + raise ImportError("AudioEncoderEncode not found") + ae = ae_cls() + encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) + + # --- 提示词 --- + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + # --- InfiniteTalk 底座 --- + prev = b.get("_previous_frames") + + # 🛡️ 总线模式：直接传递全局帧偏移量给底层，彻底废弃巨型 dummy tensor + # _process_infinite_talk_audio 现在接收 global_frame_offset 参数， + # 不再依赖 previous_frames.shape[0] 来反推音频偏移 + global_frame_offset = None + if bus_audio_mode: + global_frame_offset = b["_accumulated_frames"] + + core = XB_WanInfiniteTalkToVideo_Single() + + # � _process_infinite_talk_audio 内部已通过 model.clone() 保护原模型， + # 无需手动 save/restore——异常中断时 clone 自动被 GC 回收 + model_out, pos, neg, latent, trim = core.process( + model=b["model"], model_patch=b["model_patch"], + positive=pos_cond, negative=neg_cond, vae=b["vae"], + width=b["width"], height=b["height"], length=actual_length, + audio_encoder_output_1=encoded_audio, + motion_frame_count=motion, audio_scale=b["audio_scale"], + vae_tile_size=b["vae_encode_tile_size"], + clip_vision_output=b.get("clip_vision_output"), + start_image=b.get("start_image"), previous_frames=prev, + segment_audio=use_segment_audio, + global_frame_offset=global_frame_offset, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + # --- 采样 --- + latent_out, = nodes.KSampler().sample( + model=model_out, seed=b["seed"], + steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0, + ) + + # --- 解码 --- + decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) + + # --- 视频裁剪累加 --- + is4d = len(decoded.shape) == 4 + cd = 0 if is4d else 1 + total = decoded.shape[cd] + b["_previous_frames"] = decoded + + trim_val = int(trim) if trim else 0 + trim_val = min(trim_val, total - 1) if trim_val < total else 0 + cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] + final_video = _safe_video_accumulate(prev_video, cut, cd, label="TalkRelay") + + # --- 音频裁剪累加 --- + ats = trim_val / fps if trim_val else 0.0 + if bus_audio_mode: + # 总线模式：从原始长音频中裁出当前段 + raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) + raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 + seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) + seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) + seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) + cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None + else: + cut_audio = raw_audio + if ats > 0 and raw_audio: + wf = raw_audio["waveform"] + sr = raw_audio["sample_rate"] + n = int(ats * sr) + if n < wf.shape[-1]: + cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} + else: + cut_audio = None # 裁切长度超出自身，彻底丢弃防止音画不同步 + + if prev_audio is None: + final_audio = cut_audio + elif cut_audio is None: + final_audio = prev_audio + else: + pa, ca = prev_audio, cut_audio + if pa["sample_rate"] != ca["sample_rate"]: + if torchaudio is None: + raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") + ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) + ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} + + # 🛡️ 设备对齐 + 声道对齐: 防止 torch.cat 因设备/声道不一致崩溃 + pa_wf = pa["waveform"] + ca_wf = ca["waveform"].to(pa_wf.device) + if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: + # 🔧 严格针对 Channel 维度 (shape[-2]) 对齐 [B, C, T] 格式 + if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: + rpt = [1] * pa_wf.dim() + rpt[-2] = ca_wf.shape[-2] + pa_wf = pa_wf.repeat(*rpt) + elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: + rpt = [1] * ca_wf.dim() + rpt[-2] = pa_wf.shape[-2] + ca_wf = ca_wf.repeat(*rpt) + + final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), + "sample_rate": pa["sample_rate"]} + + b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] + if bus_audio_mode: + b["_accumulated_frames"] += cut.shape[cd] + remaining_after = b["_total_frames"] - b["_accumulated_frames"] + print(f"🎵 [XB-BOX] Relay done: {total}→{cut.shape[cd]}f (trim {trim_val}), " + f"acc={final_video.shape[cd]}f, 总线进度 {b['_accumulated_frames']}/{b['_total_frames']} (剩余 {remaining_after})") + else: + print(f"🎵 [XB-BOX] Relay done: {total}→{cut.shape[cd]}f (trim {trim_val}), " + f"acc={final_video.shape[cd]}f, video_dur={final_video.shape[cd]/fps:.1f}s") + + _refresh_models() + return (b, final_video, final_audio) + +# ============================================================ +# NEW 无限接力节点（带重叠帧数/拼接模式等新功能） +# ============================================================ + +class XB_Wan_InfiniteRelayNode_New: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "wan_bus": ("WAN_BUS",), + "start_image": ("IMAGE",), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the specific action for this segment..."}), + "trim_head_frames": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1, "tooltip": "接力重叠帧数（去重）"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定：本节点自动循环 N 次 = 串联 N 个接力点"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",) + } + } + + @classmethod + def VALIDATE_INPUTS(cls, **kwargs): + return True + + RETURN_TYPES = ("WAN_BUS", "IMAGE", "IMAGE") + RETURN_NAMES = ("📦 WAN_BUS (Pass to next)", "🖼️ Current End Image (Connect to next Start Image)", "🎞️ Accumulated Video Stream") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_bus, start_image, positive_prompt, trim_head_frames=1, relay_count=1, total_frames_display="", prev_video=None): + print(f"\n🏃‍♀️ [XB-BOX] Executing Image-to-Video Infinite pipeline × {relay_count} relay(s)...") + b = wan_bus + segment_len = b["length"] + cut_first_frame = b.get("cut_first_frame", True) + # 估算总计帧数 + est_per_segment = segment_len - (trim_head_frames if cut_first_frame and segment_len > 1 else 0) + est_total = est_per_segment * relay_count + print(f"📊 [XB-BOX] 每段约 {est_per_segment} 帧 (原始 {segment_len}), 总计约 {est_total} 帧") + + current_image = start_image + accumulated_video = prev_video + + for r in range(relay_count): + seed = b["seed"] + print(f"\n🔄 [XB-BOX] 接力 {r+1}/{relay_count}...") + + encode_tile = b.get("vae_encode_tile_size", b.get("vae_tile_size", 64)) + decode_tile = b.get("vae_decode_tile_size", b.get("vae_tile_size", 64)) + + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + cv_output = b.get("clip_vision_output") + pos, neg, latent = XB_WanImageToVideo().process( + positive=pos_cond, negative=neg_cond, vae=b["vae"], + clip_vision_output=cv_output, start_image=current_image, + width=b["width"], height=b["height"], length=segment_len, + batch_size=1, vae_tile_size=encode_tile, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + total_steps = b["steps"] + high_steps = b["high_noise_steps"] + if high_steps >= total_steps: + high_steps = total_steps - 1 + + latent_high, = nodes.KSamplerAdvanced().sample( + model=b["model_high"], add_noise="enable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, + start_at_step=0, end_at_step=high_steps, return_with_leftover_noise="enable" + ) + + latent_low, = nodes.KSamplerAdvanced().sample( + model=b["model_low"], add_noise="disable", noise_seed=seed, + steps=total_steps, cfg=b["cfg"], sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent_high, + start_at_step=high_steps, end_at_step=total_steps, return_with_leftover_noise="disable" + ) + + decoded_image, = nodes.VAEDecode().decode(samples=latent_low, vae=b["vae"]) + + is_4d = len(decoded_image.shape) == 4 + frame_dim = 0 if is_4d else 1 + total_frames = decoded_image.shape[frame_dim] + + if cut_first_frame and total_frames > 1: + safe_trim = min(trim_head_frames, total_frames - 1) + if safe_trim > 0: + if is_4d: + decoded_image = decoded_image[safe_trim:, :, :, :] + else: + decoded_image = decoded_image[:, safe_trim:, :, :, :] + + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + + decoded_image = _apply_progressive_color_correction(decoded_image, current_image, is_4d) + last_frame = decoded_image[-1:, :, :, :] if is_4d else decoded_image[:, -1:, :, :, :] + + concat_dim = 0 if is_4d else 1 + accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"InfiniteRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) + current_image = last_frame + + _refresh_models() + print(f"✅ [XB-BOX] 接力完成: {relay_count} 段, 总计 {accumulated_video.shape[concat_dim]} 帧") + return (wan_bus, current_image, accumulated_video) + + +# ============================================================ +# XB_Video_Merger — 视频拼接器 +# ============================================================ +class XB_Video_Merger: + @classmethod + def INPUT_TYPES(cls): + inputs = {"required": {}, "optional": {}} + for i in range(1, 11): + inputs["optional"][f"video_{i}"] = ("IMAGE",) + return inputs + + RETURN_TYPES = ("IMAGE",) + RETURN_NAMES = ("🎞️ Full Long Video",) + FUNCTION = "merge_videos" + CATEGORY = "XB_ToolBox/Pipeline" + + def merge_videos(self, **kwargs): + videos = [kwargs.get(f"video_{i}") for i in range(1, 11) if kwargs.get(f"video_{i}") is not None] + if not videos: + raise ValueError("🚨 [XB-BOX] Merge failed: At least one video must be connected!") + + concat_dim = 0 if len(videos[0].shape) == 4 else 1 + ref_shape = list(videos[0].shape) + for i, vid in enumerate(videos): + for d in range(len(ref_shape)): + if d != concat_dim and vid.shape[d] != ref_shape[d]: + raise ValueError( + f"🚨 [XB-BOX] Merge failed: Video segment {i+1} shape {list(vid.shape)} " + f"does not match first segment shape {ref_shape} on dimension {d} " + f"(expected {ref_shape[d]}, got {vid.shape[d]})" + ) + # 将全部视频段卸载到 CPU 再拼接，防止多段同时驻留 VRAM 导致 OOM + videos_cpu = [v.cpu() for v in videos] + merged = torch.cat(videos_cpu, dim=concat_dim) + print(f"💾 [XB-BOX] Merger: {len(videos)} 段视频已在 CPU 完成拼接 (shape={list(merged.shape)})") + return (merged,) + +# ============================================================ +# XB_StoryboardSlicer — 分镜切片器 +# ============================================================ +class XB_StoryboardSlicer: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "image": ("IMAGE",), + "mode": (["3-Grid (1x3 Horizontal)", "3-Grid (3x1 Vertical)", "4-Grid (2x2)", "6-Grid (2x3 Horizontal)", "6-Grid (3x2 Vertical)", "9-Grid (3x3)"], {"default": "4-Grid (2x2)"}), + "crop_margin": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 0.45, "step": 0.01}), + } + } + RETURN_TYPES = ("IMAGE",) * 9 + RETURN_NAMES = ("Img1", "Img2", "Img3", "Img4", "Img5", "Img6", "Img7", "Img8", "Img9") + FUNCTION = "slice_image" + CATEGORY = "XB_ToolBox/Pipeline" + + def slice_image(self, image, mode, crop_margin): + if "1x3" in mode: rows, cols = 1, 3 + elif "3x1" in mode: rows, cols = 3, 1 + elif "2x2" in mode: rows, cols = 2, 2 + elif "2x3" in mode: rows, cols = 2, 3 + elif "3x2" in mode: rows, cols = 3, 2 + elif "3x3" in mode: rows, cols = 3, 3 + else: rows, cols = 2, 2 + + batch_size, h, w, channels = image.shape + slice_h, slice_w = h // rows, w // cols + crop_y, crop_x = int(slice_h * crop_margin), int(slice_w * crop_margin) + + sliced_images = [] + for r in range(rows): + for c in range(cols): + sy, ey = r * slice_h + crop_y, (r + 1) * slice_h - crop_y + sx, ex = c * slice_w + crop_x, (c + 1) * slice_w - crop_x + if sy >= ey: ey = sy + 1 + if sx >= ex: ex = sx + 1 + sliced_images.append(image[:, sy:ey, sx:ex, :]) + + placeholder_h = max(1, slice_h - crop_y * 2) + placeholder_w = max(1, slice_w - crop_x * 2) + while len(sliced_images) < 9: + sliced_images.append(torch.zeros((1, placeholder_h, placeholder_w, 3), dtype=torch.float32, device=image.device)) + return tuple(sliced_images) + + +# ============================================================ +# XB_WanAnimate_ParamBus — Wan Animate 动作迁移参数总线 +# ============================================================ +class XB_WanAnimate_RelayNode_New: + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] + files = sorted(files) + + if not files: + files = ["[Folder empty_Please connect or upload]"] + + return { + "required": { + "wan_animate_bus": ("WAN_ANIMATE_BUS",), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), + "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), + "ref_image_file": (files, {"image_upload": True}), + "continue_motion_max_frames": ("INT", {"default": 5, "min": 0, "max": 16, "tooltip": "接力重叠帧数（衔接过渡）"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",), + } + } + + @classmethod + def VALIDATE_INPUTS(cls, **kwargs): + return True + + RETURN_TYPES = ("WAN_ANIMATE_BUS", "IMAGE") + RETURN_NAMES = ("📦 WAN_ANIMATE_BUS (传给下段)", "🎞️ 累加视频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_animate_bus, segment_length, positive_prompt="", use_local_ref_image="继承总线全局图", ref_image_file="", continue_motion_max_frames=5, relay_count=1, total_frames_display="", prev_video=None, opt_ref_image=None): + b = wan_animate_bus.copy() + print(f"\n🏃‍♀️ [XB-BOX] Executing Wan Animate Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") + accumulated_video = prev_video + + completed = 0 + for r in range(relay_count): + current_offset = b.get("current_offset", 0) + seed = b.get("seed", 123456789) + print(f"\n🔄 [XB-BOX] Animate 接力 {r+1}/{relay_count} (offset={current_offset})...") + + source_video = b.get("pose_video") if b.get("pose_video") is not None else b.get("face_video") + total_source_frames = float('inf') + bus_total = b.get("total_frames", 0) + if bus_total > 0: + total_source_frames = bus_total + elif source_video is not None: + total_source_frames = source_video.shape[0] if len(source_video.shape) == 4 else source_video.shape[1] + + remaining_frames = max(0, total_source_frames - current_offset) + + if remaining_frames <= 0: + break + + if remaining_frames < segment_length: + if current_offset == 0: + need_raw = remaining_frames + else: + need_raw = remaining_frames + continue_motion_max_frames + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + break + print(f"⚠️ [XB-BOX] 最后一段: 剩余 {remaining_frames} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + is_local = (use_local_ref_image in ("独立参考图", True, 1)) + if is_local: + if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": + print("⏭️ [XB-BOX] 独立参考图未选择，跳过") + break + image_path = folder_paths.get_annotated_filepath(ref_image_file) + if not os.path.exists(image_path): + print(f"⏭️ [XB-BOX] 参考图文件未找到，跳过") + break + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + image_data = i.convert("RGB") + image_data = np.array(image_data).astype(np.float32) / 255.0 + ref_image = torch.from_numpy(image_data)[None,] + cv_model = b.get("clip_vision") + if cv_model is None: + raise ValueError("🚨 独立参考图需要 clip_vision 模型！") + w, h = b.get("width", 480), b.get("height", 832) + sm = b.get("scale_method", "lanczos") + cm = b.get("crop_mode", "center") + clip_img = comfy.utils.common_upscale(ref_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) + clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") + else: + ref_image = b.get("global_ref_image") + if ref_image is None: + raise ValueError("🚨 缺少参考图！") + clip_vision_output = b.get("clip_vision_output") + + prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) + + continue_motion = None + cont_max_frames = continue_motion_max_frames + if is_local: + cont_max_frames = 0 + if accumulated_video is not None and cont_max_frames > 0: + is_4d_v = len(accumulated_video.shape) == 4 + if is_4d_v: + continue_motion = accumulated_video[-cont_max_frames:, :, :, :] + else: + continue_motion = accumulated_video[:, -cont_max_frames:, :, :, :] + + try: + from .nodes_wan import XB_WanAnimateToVideo + except ImportError: + raise ImportError("🚨 找不到 XB_WanAnimateToVideo") + + animate_core = XB_WanAnimateToVideo() + func_name = getattr(XB_WanAnimateToVideo, "FUNCTION", "process") + func = getattr(animate_core, func_name) + kwargs = { + "positive": pos_cond, "negative": neg_cond, "vae": b["vae"], + "clip_vision_output": clip_vision_output, "reference_image": ref_image, + "face_video": b.get("face_video"), "pose_video": b.get("pose_video"), + "background_video": b.get("background_video"), "character_mask": b.get("character_mask"), + "continue_motion": continue_motion, + "width": b["width"], "height": b["height"], "length": actual_length, "batch_size": 1, + "continue_motion_max_frames": cont_max_frames, + "video_frame_offset": current_offset, + "vae_tile_size": b.get("vae_encode_tile_size", 320) + } + pos, neg, latent, trim_latent, trim_image, new_offset = func(**kwargs) + + latent_sampled, = nodes.KSampler().sample( + model=b["model"], seed=seed, steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0 + ) + trim_latent_val = trim_latent if trim_latent is not None else 0 + if trim_latent_val > 0: + latent_sampled["samples"] = latent_sampled["samples"][:, :, trim_latent_val:, :, :] + + try: + from ..core.nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal + except ImportError: + raise ImportError("🚨 找不到 _AliasVAEDecodeTemporal") + cleanup_mode = b.get("cleanup", "不做任何清理") + decoder = XB_ROCmVAEDecodeTemporal() + decoded_image, = decoder.decode( + samples=latent_sampled, vae=b["vae"], + tile=b.get("vae_decode_tile_size", 320), overlap=b.get("spatial_overlap", 32), + t_tile=b.get("temporal_chunk_size", 64), t_overlap=b.get("temporal_overlap", 8), + cleanup=cleanup_mode + ) + + is_4d = len(decoded_image.shape) == 4 + concat_dim = 0 if is_4d else 1 + trim_px = int(trim_image) if trim_image else 0 + if trim_px > 0 and trim_px < decoded_image.shape[concat_dim]: + if is_4d: + decoded_image = decoded_image[trim_px:, :, :, :] + else: + decoded_image = decoded_image[:, trim_px:, :, :, :] + + accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"AnimateRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) + actual_out = decoded_image.shape[concat_dim] + b["current_offset"] = current_offset + actual_out + print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={actual_out}f total={accumulated_video.shape[concat_dim]}f") + completed = r + 1 + + if completed < relay_count: + print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") + else: + print(f"🎉 [XB-BOX] Animate 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") + overlap = continue_motion_max_frames + total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) + return (b, accumulated_video) + + +# ============================================================ +# XB_WanInfiniteTalk_ParamBus — InfiniteTalk 无限对口型参数总线 +# ============================================================ +class XB_WanInfiniteTalk_RelayNode_New: + """ + 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 + 音频独立输入 + 自动裁剪重叠 + 累加输出，与视频帧对齐。 + relay_count > 1 时本节点内部自动循环 N 次。 + """ + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), + "positive_prompt": ("STRING", {"multiline": True, "default": ""}), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",), + "prev_audio": ("AUDIO",), + "audio": ("AUDIO",), + } + } + + RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") + RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, motion_frame_count=9, relay_count=1, total_frames_display="", + prev_video=None, prev_audio=None, audio=None): + b = wan_infinitetalk_bus.copy() + fps = b.get("fps", 25.0) + bus_audio_mode = b.get("_bus_audio_mode", False) + print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") + + accumulated_video = prev_video + accumulated_audio = prev_audio + + completed = 0 + for r in range(relay_count): + print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") + + # ================================================================ + # 🛡️ 模式判断 & 帧数计算 + # ================================================================ + if bus_audio_mode: + # --- 模式1：总线音频模式 --- + total_frames = b["_total_frames"] + accumulated = b["_accumulated_frames"] + remaining = total_frames - accumulated + + if remaining <= 0: + break + + if remaining < segment_length: + if accumulated == 0: + need_raw = remaining # 第一段无 trim + else: + need_raw = remaining + motion_frame_count + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + break + print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + encoded_audio = b["_encoded_audio"] + raw_audio = b.get("_raw_audio") + use_segment_audio = False + print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") + else: + # --- 模式2：独立音频模式 --- + if audio is None: + print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") + break + + actual_length = segment_length + encoded_audio = None + raw_audio = audio + use_segment_audio = True + print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") + + # --- 🛡️ 帧数对齐：Wan 模型要求 length = 4n+1 --- + raw_length = actual_length + aligned_length = ((actual_length + 2) // 4) * 4 + 1 + if aligned_length != raw_length: + print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") + if not bus_audio_mode and audio is not None: + pad_frames = aligned_length - raw_length + pad_sec = pad_frames / fps + sr = audio["sample_rate"] + pad_samples = int(pad_sec * sr) + pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) + raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} + actual_length = aligned_length + + # --- 音频编码（独立模式） --- + if not bus_audio_mode: + ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") + if ae_cls is None: + raise ImportError("AudioEncoderEncode not found") + ae = ae_cls() + encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) + + # --- 提示词 --- + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + # --- InfiniteTalk 底座 --- + prev = b.get("_previous_frames") + global_frame_offset = None + if bus_audio_mode: + global_frame_offset = b["_accumulated_frames"] + + core = XB_WanInfiniteTalkToVideo_Single() + model_out, pos, neg, latent, trim = core.process( + model=b["model"], model_patch=b["model_patch"], + positive=pos_cond, negative=neg_cond, vae=b["vae"], + width=b["width"], height=b["height"], length=actual_length, + audio_encoder_output_1=encoded_audio, + motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], + vae_tile_size=b["vae_encode_tile_size"], + clip_vision_output=b.get("clip_vision_output"), + start_image=b.get("start_image"), previous_frames=prev, + segment_audio=use_segment_audio, + global_frame_offset=global_frame_offset, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + # --- 采样 --- + latent_out, = nodes.KSampler().sample( + model=model_out, seed=b["seed"], + steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0, + ) + + # --- 解码 --- + decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) + + # --- 视频裁剪累加 --- + is4d = len(decoded.shape) == 4 + cd = 0 if is4d else 1 + total = decoded.shape[cd] + b["_previous_frames"] = decoded + + trim_val = int(trim) if trim else 0 + trim_val = min(trim_val, total - 1) if trim_val < total else 0 + cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] + final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkRelay-{r+1}", concat_mode=b.get("concat_mode", "自动")) + + # --- 音频裁剪累加 --- + ats = trim_val / fps if trim_val else 0.0 + if bus_audio_mode: + raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) + raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 + seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) + seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) + seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) + cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None + else: + cut_audio = raw_audio + if ats > 0 and raw_audio: + wf = raw_audio["waveform"] + sr = raw_audio["sample_rate"] + n = int(ats * sr) + if n < wf.shape[-1]: + cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} + else: + cut_audio = None + + if accumulated_audio is None: + final_audio = cut_audio + elif cut_audio is None: + final_audio = accumulated_audio + else: + pa, ca = accumulated_audio, cut_audio + if pa["sample_rate"] != ca["sample_rate"]: + if torchaudio is None: + raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") + ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) + ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} + pa_wf = pa["waveform"] + ca_wf = ca["waveform"].to(pa_wf.device) + if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: + if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: + rpt = [1] * pa_wf.dim() + rpt[-2] = ca_wf.shape[-2] + pa_wf = pa_wf.repeat(*rpt) + elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: + rpt = [1] * ca_wf.dim() + rpt[-2] = pa_wf.shape[-2] + ca_wf = ca_wf.repeat(*rpt) + final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} + + b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] + if bus_audio_mode: + b["_accumulated_frames"] += cut.shape[cd] + accumulated_video = final_video + accumulated_audio = final_audio + print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f total={final_video.shape[cd]}f") + completed = r + 1 + + if completed < relay_count: + print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") + else: + print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") + _refresh_models() + overlap = motion_frame_count + total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) + return (b, accumulated_video, accumulated_audio) + + +# ============================================================ +# XB_WanSCAIL_ParamBus — SCAIL 无限时长参数总线 +# ============================================================ +class XB_WanSCAIL_ParamBus_New: + """SCAIL 无限接力参数总线。 + 集中管理模型、姿态视频、参考图等所有共享参数， + 接力点只需传正面提示词 + 单次生成帧数，其余全从总线走。 + """ + _SCALE_METHODS = ["bilinear", "bicubic", "lanczos", "nearest-exact", "area"] + _CROP_MODES = ["center", "disabled"] + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + # --- 模型 --- + "model": ("MODEL",), + "clip": ("CLIP",), + "vae": ("VAE",), + "clip_vision": ("CLIP_VISION",), + # --- 基础参数 --- + "negative_prompt": ("STRING", {"multiline": True, "default": "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"}), + "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), + "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), + "total_frames": ("INT", {"default": 0, "min": 0, "max": 999999, "tooltip": "0=自动从姿态视频取长度；>0=强制截断到此帧数"}), + "fps": ("FLOAT", {"default": 16.0, "min": 1.0, "max": 120.0, "step": 1.0}), + # --- SCAIL 控制 --- + "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "replacement_mode": ("BOOLEAN", {"default": False, "label_on": "替换模式", "label_off": "动画模式"}), + "concat_mode": (["自动", "强制GPU", "强制CPU"], {"default": "自动", "tooltip": "视频累积拼接位置: 自动=显存>30%切CPU | 强制GPU=始终GPU | 强制CPU=始终CPU"}), + # --- VAE 分块 --- + "vae_encode_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + "vae_decode_tile_size": ("INT", {"default": 192, "min": 64, "max": 3840, "step": 32}), + "spatial_overlap": ("INT", {"default": 32, "min": 0, "max": 3840, "step": 32}), + "temporal_chunk_size": ("INT", {"default": 64, "min": 0, "max": 8192, "step": 4}), + "temporal_overlap": ("INT", {"default": 8, "min": 0, "max": 8192, "step": 4}), + # --- 采样 --- + "steps": ("INT", {"default": 20, "min": 1, "max": 100}), + "cfg": ("FLOAT", {"default": 5.0, "min": 0.0, "max": 10.0, "step": 0.1}), + "sampler_name": (comfy.samplers.KSAMPLER_NAMES,), + "scheduler": (comfy.samplers.SCHEDULER_NAMES,), + "seed": ("INT", {"default": 123456789}), + # --- 清理 --- + "cleanup": (["不做任何清理", "卸载显存模型", "卸载全量模型"], {"default": "不做任何清理"}), + }, + "optional": { + "global_ref_image": ("IMAGE",), + "reference_image_mask": ("IMAGE", {"tooltip": "参考图遮罩 (替换模式: 隔离角色区域)"}), + "pose_video": ("IMAGE", {"tooltip": "姿态/驱动视频"}), + "pose_video_mask": ("IMAGE", {"tooltip": "SCAIL-2 彩色遮罩视频"}), + "scale_method": (cls._SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (cls._CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("WAN_SCAIL_BUS",) + RETURN_NAMES = ("📦 WAN_SCAIL_BUS",) + FUNCTION = "pack_bus" + CATEGORY = "XB_ToolBox/Pipeline" + + def pack_bus(self, **kwargs): + # CLIP 视觉编码：参考图缩放到视频尺寸后编码 + cv = kwargs.get("clip_vision") + ref = kwargs.get("global_ref_image") + if cv is not None and ref is not None: + w, h = kwargs["width"], kwargs["height"] + method = kwargs.get("scale_method", "lanczos") + crop = kwargs.get("crop_mode", "center") + scaled = comfy.utils.common_upscale(ref.movedim(-1, 1), w, h, method, crop).movedim(1, -1) + kwargs["clip_vision_output"], = nodes.CLIPVisionEncode().encode(cv, scaled, "center") + print(f"👁️ [XB-BOX] SCAIL CLIP视觉编码 ({ref.shape[1]}×{ref.shape[2]} → {w}×{h}, {method}/{crop})") + else: + kwargs["clip_vision_output"] = None + + kwargs["current_offset"] = 0 + return (kwargs,) + + +# ============================================================ +# XB_WanSCAIL_RelayNode — SCAIL 无限接力点 +# ============================================================ +class XB_WanSCAIL_RelayNode_New: + """SCAIL 无限接力节点。 + 每个接力点完成：CLIPTextEncode → SCAIL Process → KSampler → VAEDecode → trim → 累加。 + 自动管理姿态视频偏移 + 前段尾帧衔接。 + relay_count > 1 时本节点内部自动循环 N 次。 + """ + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "wan_scail_bus": ("WAN_SCAIL_BUS",), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "positive_prompt": ("STRING", {"multiline": True, "default": "Describe the scene..."}), + "previous_frame_count": ("INT", {"default": 5, "min": 1, "max": 8192, "step": 4, "tooltip": "接力重叠帧数（尾帧衔接）"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",), + } + } + + RETURN_TYPES = ("WAN_SCAIL_BUS", "IMAGE") + RETURN_NAMES = ("📦 WAN_SCAIL_BUS (传给下段)", "🎞️ 累加视频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_scail_bus, segment_length, positive_prompt="", previous_frame_count=5, relay_count=1, total_frames_display="", prev_video=None): + b = wan_scail_bus.copy() + print(f"\n🏃‍♀️ [XB-BOX] Executing SCAIL Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 总计约 {segment_length * relay_count} 帧") + accumulated_video = prev_video + + completed = 0 + for r in range(relay_count): + current_offset = b.get("current_offset", 0) + seed = b.get("seed", 123456789) + print(f"\n🔄 [XB-BOX] SCAIL 接力 {r+1}/{relay_count} (offset={current_offset})...") + + pose_video = b.get("pose_video") + total_source_frames = float('inf') + bus_total = b.get("total_frames", 0) + if bus_total > 0: + total_source_frames = bus_total + elif pose_video is not None: + total_source_frames = pose_video.shape[0] if len(pose_video.shape) == 4 else pose_video.shape[1] + + remaining_frames = max(0, total_source_frames - current_offset) + + if remaining_frames <= 0: + break + + if remaining_frames < segment_length: + if current_offset == 0: + need_raw = remaining_frames + else: + need_raw = remaining_frames + previous_frame_count + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + break + print(f"⚠️ [XB-BOX] SCAIL 最后一段: 剩余 {remaining_frames} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + # --- 参考图：始终从总线继承（已含遮罩处理） --- + ref_image = b.get("global_ref_image") + clip_vision_output = b.get("clip_vision_output") + + # --- 提示词 --- + prompt_text = positive_prompt.strip() if positive_prompt.strip() else b.get("positive_prompt", "") + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], prompt_text) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b.get("negative_prompt", "")) + + # --- 准备 previous_frames --- + previous_frames = None + if accumulated_video is not None and current_offset > 0: + is_4d = len(accumulated_video.shape) == 4 + if is_4d: + previous_frames = accumulated_video[-previous_frame_count:] + else: + previous_frames = accumulated_video[:, -previous_frame_count:] + + # --- SCAIL Pro 底座 --- + try: + from .nodes_wan_vae import XB_WanSCAILToVideoPro + except ImportError: + raise ImportError("🚨 找不到 XB_WanSCAILToVideoPro") + scail_core = XB_WanSCAILToVideoPro() + pos, neg, latent, new_offset = scail_core.process( + positive=pos_cond, negative=neg_cond, vae=b["vae"], + width=b["width"], height=b["height"], length=actual_length, batch_size=1, + pose_strength=b.get("pose_strength", 1.0), + pose_start=b.get("pose_start", 0.0), pose_end=b.get("pose_end", 1.0), + replacement_mode=b.get("replacement_mode", False), + video_frame_offset=current_offset, previous_frame_count=previous_frame_count, + vae_tile_size=b.get("vae_encode_tile_size", 256), + pose_video=pose_video, pose_video_mask=b.get("pose_video_mask"), + reference_image=ref_image, reference_image_mask=b.get("reference_image_mask"), + clip_vision_output=clip_vision_output, previous_frames=previous_frames, + ) + + # --- 采样 --- + latent_sampled, = nodes.KSampler().sample( + model=b["model"], seed=seed, steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0, + ) + + # --- 解码 --- + try: + from ..core.nodes_vanilla_wrappers import _AliasVAEDecodeTemporal as XB_ROCmVAEDecodeTemporal + except ImportError: + raise ImportError("🚨 找不到 _AliasVAEDecodeTemporal") + cleanup_mode = b.get("cleanup", "不做任何清理") + decoder = XB_ROCmVAEDecodeTemporal() + decoded_image, = decoder.decode( + samples=latent_sampled, vae=b["vae"], + tile=b.get("vae_decode_tile_size", 192), overlap=b.get("spatial_overlap", 32), + t_tile=b.get("temporal_chunk_size", 64), t_overlap=b.get("temporal_overlap", 8), + cleanup=cleanup_mode, + ) + + # --- 裁剪重叠帧 & 累加 --- + is_4d = len(decoded_image.shape) == 4 + concat_dim = 0 if is_4d else 1 + total_out = decoded_image.shape[concat_dim] + trim_px = previous_frame_count if (accumulated_video is not None and current_offset > 0) else 0 + trim_px = min(trim_px, total_out - 1) if trim_px < total_out else 0 + if trim_px > 0: + if is_4d: + decoded_image = decoded_image[trim_px:, :, :, :] + else: + decoded_image = decoded_image[:, trim_px:, :, :, :] + + accumulated_video = _safe_video_accumulate(accumulated_video, decoded_image, concat_dim, label=f"SCAIL-{r+1}", concat_mode=b.get("concat_mode", "自动")) + actual_out = decoded_image.shape[concat_dim] + b["current_offset"] = new_offset + print(f"✅ [XB-BOX] SCAIL 接力 {r+1}/{relay_count} 完成. out={actual_out}f total={accumulated_video.shape[concat_dim]}f") + completed = r + 1 + + if completed < relay_count: + print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") + else: + print(f"🎉 [XB-BOX] SCAIL 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") + _refresh_models() + overlap = previous_frame_count + total = max(0, segment_length * relay_count - overlap * (relay_count - 1)) + return (b, accumulated_video) + + +# ============================================================ +# XB_WanInfiniteTalk_RelayNode_MultiRef — InfiniteTalk 多图接力点 +# ============================================================ +class XB_WanInfiniteTalk_RelayNode_MultiRef: + """ + InfiniteTalk 无限对口型接力点（多图版）。 + 支持独立参考图上传，类似 Animate 接力点。 + 每个接力点完成：CLIPTextEncode → WanInfiniteTalkToVideo_Single → KSampler → VAEDecode → trim → 累加。 + relay_count > 1 时本节点内部自动循环 N 次。 + """ + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] + files = sorted(files) + if not files: + files = ["[Folder empty_Please connect or upload]"] + + return { + "required": { + "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), + "positive_prompt": ("STRING", {"multiline": True, "default": ""}), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "use_local_ref_image": (["继承总线全局图", "独立参考图"], {"default": "继承总线全局图"}), + "ref_image_file": (files, {"image_upload": True}), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",), + "prev_audio": ("AUDIO",), + "audio": ("AUDIO",), + } + } + + RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") + RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, use_local_ref_image="继承总线全局图", ref_image_file="", + motion_frame_count=9, relay_count=1, total_frames_display="", + prev_video=None, prev_audio=None, audio=None): + b = wan_infinitetalk_bus.copy() + fps = b.get("fps", 25.0) + bus_audio_mode = b.get("_bus_audio_mode", False) + print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk MultiRef Relay × {relay_count} relay(s)... 每段 {segment_length} 帧") + + accumulated_video = prev_video + accumulated_audio = prev_audio + + # --- 预处理独立参考图 --- + is_local = (use_local_ref_image in ("独立参考图", True, 1)) + local_start_image = None + local_clip_vision_output = None + if is_local: + if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": + raise ValueError("🚨 [XB-BOX] 多图接力点已开启独立参考图，但未选择图片文件！") + image_path = folder_paths.get_annotated_filepath(ref_image_file) + if not os.path.exists(image_path): + raise ValueError(f"🚨 [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}") + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + image_data = i.convert("RGB") + image_data = np.array(image_data).astype(np.float32) / 255.0 + local_start_image = torch.from_numpy(image_data)[None,] + cv_model = b.get("clip_vision") + if cv_model is None: + raise ValueError("🚨 [XB-BOX] 独立参考图需要总线连接 clip_vision 模型！") + w, h = b["width"], b["height"] + sm = b.get("scale_method", "lanczos") + cm = b.get("crop_mode", "center") + clip_img = comfy.utils.common_upscale(local_start_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) + local_clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") + print(f"🖼️ [XB-BOX] 多图接力点使用独立参考图 ({image_data.shape[1]}×{image_data.shape[0]} → {w}×{h}, {sm}/{cm})") + + completed = 0 + for r in range(relay_count): + print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") + + if bus_audio_mode: + total_frames = b["_total_frames"] + accumulated = b["_accumulated_frames"] + remaining = total_frames - accumulated + + if remaining <= 0: + break + + if remaining < segment_length: + if accumulated == 0: + need_raw = remaining + else: + need_raw = remaining + motion_frame_count + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + break + print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + encoded_audio = b["_encoded_audio"] + raw_audio = b.get("_raw_audio") + use_segment_audio = False + print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") + else: + if audio is None: + print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") + break + + actual_length = segment_length + encoded_audio = None + raw_audio = audio + use_segment_audio = True + print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") + + # --- 帧数对齐 4N+1 --- + raw_length = actual_length + aligned_length = ((actual_length + 2) // 4) * 4 + 1 + if aligned_length != raw_length: + print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") + if not bus_audio_mode and audio is not None: + pad_frames = aligned_length - raw_length + pad_sec = pad_frames / fps + sr = audio["sample_rate"] + pad_samples = int(pad_sec * sr) + pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) + raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} + actual_length = aligned_length + + # --- 音频编码（独立模式） --- + if not bus_audio_mode: + ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") + if ae_cls is None: + raise ImportError("AudioEncoderEncode not found") + ae = ae_cls() + encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) + + # --- 提示词 --- + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + # --- 参考图/CLIP视觉 --- + prev = b.get("_previous_frames") + global_frame_offset = None + if bus_audio_mode: + global_frame_offset = b["_accumulated_frames"] + + # 独立参考图：每个接力段使用独立编码；总线参考图：从总线获取 + if is_local: + cur_start_image = local_start_image + cur_clip_output = local_clip_vision_output + else: + cur_start_image = b.get("start_image") + cur_clip_output = b.get("clip_vision_output") + + # --- InfiniteTalk 底座 --- + core = XB_WanInfiniteTalkToVideo_Single() + model_out, pos, neg, latent, trim = core.process( + model=b["model"], model_patch=b["model_patch"], + positive=pos_cond, negative=neg_cond, vae=b["vae"], + width=b["width"], height=b["height"], length=actual_length, + audio_encoder_output_1=encoded_audio, + motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], + vae_tile_size=b["vae_encode_tile_size"], + clip_vision_output=cur_clip_output, + start_image=cur_start_image, previous_frames=prev, + segment_audio=use_segment_audio, + global_frame_offset=global_frame_offset, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + # --- 采样 --- + latent_out, = nodes.KSampler().sample( + model=model_out, seed=b["seed"], + steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0, + ) + + # --- 解码 --- + decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) + + # --- 视频裁剪累加 --- + is4d = len(decoded.shape) == 4 + cd = 0 if is4d else 1 + total = decoded.shape[cd] + b["_previous_frames"] = decoded + + trim_val = int(trim) if trim else 0 + trim_val = min(trim_val, total - 1) if trim_val < total else 0 + cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] + final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkMultiRef-{r+1}", concat_mode=b.get("concat_mode", "自动")) + + # --- 音频裁剪累加 --- + ats = trim_val / fps if trim_val else 0.0 + if bus_audio_mode: + raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) + raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 + seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) + seg_end_sample = int((b["_accumulated_frames"] + cut.shape[cd] + trim_val) / fps * raw_sr) + seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) + cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None + else: + cut_audio = raw_audio + if ats > 0 and raw_audio: + wf = raw_audio["waveform"] + sr = raw_audio["sample_rate"] + n = int(ats * sr) + if n < wf.shape[-1]: + cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} + else: + cut_audio = None + + if accumulated_audio is None: + final_audio = cut_audio + elif cut_audio is None: + final_audio = accumulated_audio + else: + pa, ca = accumulated_audio, cut_audio + if pa["sample_rate"] != ca["sample_rate"]: + if torchaudio is None: + raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") + ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) + ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} + pa_wf = pa["waveform"] + ca_wf = ca["waveform"].to(pa_wf.device) + if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: + if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: + rpt = [1] * pa_wf.dim(); rpt[-2] = ca_wf.shape[-2]; pa_wf = pa_wf.repeat(*rpt) + elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: + rpt = [1] * ca_wf.dim(); rpt[-2] = pa_wf.shape[-2]; ca_wf = ca_wf.repeat(*rpt) + final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} + + b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] + if bus_audio_mode: + b["_accumulated_frames"] += cut.shape[cd] + accumulated_video = final_video + accumulated_audio = final_audio + print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f total={final_video.shape[cd]}f") + completed = r + 1 + + if completed < relay_count: + print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") + else: + print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") + # 👑 刷新总线全局状态：如果引入了独立参考图，将其升级为下游节点的新基准 + if is_local_ref: + b["start_image"] = local_start_image + b["clip_vision_output"] = local_clip_vision_output + print("🌐 [XB-BOX] 总线状态已更新：独立参考图已成为下游节点的新全局空间基准。") + + _refresh_models() + return (b, accumulated_video, accumulated_audio) + + +# ============================================================ +# XB_WanInfiniteTalk_RelayNode_AllInOne — InfiniteTalk 全能接力点 +# ============================================================ +class XB_WanInfiniteTalk_RelayNode_AllInOne: + """ + InfiniteTalk 无限对口型接力点（全能版）。 + 首帧来源三选一：独立参考图 | 继承总线全局图 | 继承前段尾帧。 + 自动对齐音频裁剪与重叠帧，确保跳帧和长度一致。 + """ + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] if os.path.exists(input_dir) else [] + files = sorted(files) + if not files: + files = ["[Folder empty_Please connect or upload]"] + + return { + "required": { + "wan_infinitetalk_bus": ("WAN_INFINITETALK_BUS",), + "positive_prompt": ("STRING", {"multiline": True, "default": ""}), + "segment_length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "start_image_mode": (["继承总线全局图", "独立参考图", "继承前段尾帧"], {"default": "继承总线全局图"}), + "ref_image_file": (files, {"image_upload": True}), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1, "tooltip": "接力重叠帧数（运动过渡）。音频和视频裁剪会自动对齐"}), + "relay_count": ("INT", {"default": 1, "min": 1, "max": 999, "step": 1, "tooltip": "接力数量设定"}), + "total_frames_display": ("STRING", {"default": "", "multiline": False, "tooltip": "总计生成帧数（自动计算）"}), + }, + "optional": { + "prev_video": ("IMAGE",), + "prev_audio": ("AUDIO",), + "audio": ("AUDIO",), + } + } + + RETURN_TYPES = ("WAN_INFINITETALK_BUS", "IMAGE", "AUDIO") + RETURN_NAMES = ("📦 WAN_INFINITETALK_BUS (传给下段)", "🎞️ 累加视频流", "🔊 累加音频流") + FUNCTION = "execute_relay" + CATEGORY = "XB_ToolBox/Pipeline" + + def execute_relay(self, wan_infinitetalk_bus, positive_prompt, segment_length, start_image_mode="继承总线全局图", ref_image_file="", + motion_frame_count=9, relay_count=1, total_frames_display="", + prev_video=None, prev_audio=None, audio=None): + b = wan_infinitetalk_bus.copy() + fps = b.get("fps", 25.0) + bus_audio_mode = b.get("_bus_audio_mode", False) + print(f"\n🏃‍♀️ [XB-BOX] Executing InfiniteTalk AllInOne Relay × {relay_count} relay(s)... 每段 {segment_length} 帧, 模式: {start_image_mode}") + + accumulated_video = prev_video + accumulated_audio = prev_audio + + # --- 预处理独立参考图 --- + is_local_ref = (start_image_mode == "独立参考图") + is_tail_frame = (start_image_mode == "继承前段尾帧") + local_start_image = None + local_clip_vision_output = None + if is_local_ref: + if not ref_image_file or ref_image_file == "[Folder empty_Please connect or upload]": + raise ValueError("🚨 [XB-BOX] 全能接力点已开启独立参考图，但未选择图片文件！") + image_path = folder_paths.get_annotated_filepath(ref_image_file) + if not os.path.exists(image_path): + raise ValueError(f"🚨 [XB-BOX] 独立参考图文件未找到: {os.path.basename(image_path)}") + i = Image.open(image_path) + i = ImageOps.exif_transpose(i) + image_data = i.convert("RGB") + image_data = np.array(image_data).astype(np.float32) / 255.0 + local_start_image = torch.from_numpy(image_data)[None,] + cv_model = b.get("clip_vision") + if cv_model is None: + raise ValueError("🚨 [XB-BOX] 独立参考图需要总线连接 clip_vision 模型！") + w, h = b["width"], b["height"] + sm = b.get("scale_method", "lanczos") + cm = b.get("crop_mode", "center") + clip_img = comfy.utils.common_upscale(local_start_image.movedim(-1, 1), w, h, sm, cm).movedim(1, -1) + local_clip_vision_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") + print(f"🖼️ [XB-BOX] 全能接力点使用独立参考图 ({image_data.shape[1]}×{image_data.shape[0]} → {w}×{h}, {sm}/{cm})") + + completed = 0 + for r in range(relay_count): + print(f"\n🔄 [XB-BOX] InfiniteTalk 接力 {r+1}/{relay_count}...") + + if bus_audio_mode: + total_frames = b["_total_frames"] + accumulated = b["_accumulated_frames"] + remaining = total_frames - accumulated + + if remaining <= 0: + break + + if remaining < segment_length: + if accumulated == 0: + need_raw = remaining + else: + need_raw = remaining + motion_frame_count + actual_length = ((need_raw + 2) // 4) * 4 + 1 + if actual_length < 1: + break + print(f"\n⚠️ [XB-BOX] 最后一段: 剩余 {remaining} → need_raw={need_raw} → {actual_length} 帧 (4N+1)") + else: + actual_length = segment_length + + encoded_audio = b["_encoded_audio"] + raw_audio = b.get("_raw_audio") + use_segment_audio = False + print(f"\n🔊 [XB-BOX] 总线音频接力: 偏移 {accumulated} 帧 @ {fps:.0f}fps, 生成长度 {actual_length} 帧 ({accumulated}/{total_frames})") + else: + if audio is None: + print(f"⏭️ [XB-BOX] 无音频输入，跳过接力 {r+1}") + break + + actual_length = segment_length + encoded_audio = None + raw_audio = audio + use_segment_audio = True + print(f"\n🎵 [XB-BOX] 独立音频接力: 生成长度 {actual_length} 帧") + + # --- 帧数对齐 4N+1 --- + raw_length = actual_length + aligned_length = ((actual_length + 2) // 4) * 4 + 1 + if aligned_length != raw_length: + print(f"⚠️ [XB-BOX] 帧数对齐: {raw_length} → {aligned_length} (4n+1)") + if not bus_audio_mode and audio is not None: + pad_frames = aligned_length - raw_length + pad_sec = pad_frames / fps + sr = audio["sample_rate"] + pad_samples = int(pad_sec * sr) + pad_wf = torch.zeros((audio["waveform"].shape[0], pad_samples), dtype=audio["waveform"].dtype, device=audio["waveform"].device) + raw_audio = {"waveform": torch.cat([audio["waveform"], pad_wf], dim=-1), "sample_rate": sr} + actual_length = aligned_length + + if not bus_audio_mode: + ae_cls = nodes.NODE_CLASS_MAPPINGS.get("AudioEncoderEncode") + if ae_cls is None: + raise ImportError("AudioEncoderEncode not found") + ae = ae_cls() + encoded_audio, = getattr(ae, ae.FUNCTION)(b["audio_encoder"], raw_audio) + + pos_cond, = nodes.CLIPTextEncode().encode(b["clip"], positive_prompt) + neg_cond, = nodes.CLIPTextEncode().encode(b["clip"], b["negative_prompt"]) + + prev = b.get("_previous_frames") + global_frame_offset = b["_accumulated_frames"] if bus_audio_mode else None + + # --- 动态决定当前段的首帧来源 --- + # 🔑 r==0 尊重用户选择; r>0 强制切换为"继承前段尾帧"，防止姿势回拉 + current_step_mode = start_image_mode if r == 0 else "继承前段尾帧" + + if current_step_mode == "独立参考图": + cur_start_image = local_start_image + cur_clip_output = local_clip_vision_output + # 💥 Hard Cut: 换参考图=镜头切换，必须清空前置重叠帧 + # 否则旧画面的运动残影会强行揉进新画面，导致 20~40 帧的"恐怖渐变" + prev = None + print("🎬 [XB-BOX] 触发镜头硬切 (Hard Cut): 已清空前置重叠帧，启用独立参考图") + + elif current_step_mode == "继承前段尾帧": + if prev is not None: + is4d_prev = len(prev.shape) == 4 + cur_start_image = prev[-1:] if is4d_prev else prev[:, -1:] + + # VAE 解码可能输出 5D，必须压缩为 4D + if not is4d_prev: + while len(cur_start_image.shape) > 4: + cur_start_image = cur_start_image[:, 0] + + cv_model = b.get("clip_vision") + if cv_model is not None: + w, h = b["width"], b["height"] + sm = b.get("scale_method", "lanczos") + cm = b.get("crop_mode", "center") + clip_input = cur_start_image.clamp(0.0, 1.0).movedim(-1, 1) + clip_img = comfy.utils.common_upscale(clip_input, w, h, sm, cm).movedim(1, -1) + cur_clip_output, = nodes.CLIPVisionEncode().encode(cv_model, clip_img, "center") + print(f"🔗 [XB-BOX] 尾帧推演: prev={prev.shape}→start={cur_start_image.shape}, clip_vision 已重新编码") + else: + cur_clip_output = b.get("clip_vision_output") + print(f"🔗 [XB-BOX] 尾帧推演: prev={prev.shape}→start={cur_start_image.shape}, 无 clip_vision") + else: + cur_start_image = b.get("start_image") + cur_clip_output = b.get("clip_vision_output") + print("⚠️ [XB-BOX] 尾帧推演无前置帧，回退到总线全局图") + + else: # "继承总线全局图" + cur_start_image = b.get("start_image") + cur_clip_output = b.get("clip_vision_output") + print("🌐 [XB-BOX] 全局图模式: 继承总线原始设定") + core = XB_WanInfiniteTalkToVideo_Single() + model_out, pos, neg, latent, trim = core.process( + model=b["model"], model_patch=b["model_patch"], + positive=pos_cond, negative=neg_cond, vae=b["vae"], + width=b["width"], height=b["height"], length=actual_length, + audio_encoder_output_1=encoded_audio, + motion_frame_count=motion_frame_count, audio_scale=b["audio_scale"], + vae_tile_size=b["vae_encode_tile_size"], + clip_vision_output=cur_clip_output, + start_image=cur_start_image, previous_frames=prev, + segment_audio=use_segment_audio, + global_frame_offset=global_frame_offset, + scale_method=b.get("scale_method", "lanczos"), + crop_mode=b.get("crop_mode", "center"), + ) + + latent_out, = nodes.KSampler().sample( + model=model_out, seed=b["seed"], + steps=b["steps"], cfg=b["cfg"], + sampler_name=b["sampler_name"], scheduler=b["scheduler"], + positive=pos, negative=neg, latent_image=latent, denoise=1.0, + ) + + decoded, = nodes.VAEDecode().decode(samples=latent_out, vae=b["vae"]) + + is4d = len(decoded.shape) == 4 + cd = 0 if is4d else 1 + total = decoded.shape[cd] + b["_previous_frames"] = decoded + + # 🛡️ 关键对齐：trim_val 来自模型返回的重叠帧数，音频和视频裁剪统一使用此值 + trim_val = int(trim) if trim else 0 + trim_val = min(trim_val, total - 1) if trim_val < total else 0 + cut = decoded[trim_val:] if is4d else decoded[:, trim_val:] + final_video = _safe_video_accumulate(accumulated_video, cut, cd, label=f"TalkAIO-{r+1}", concat_mode=b.get("concat_mode", "自动")) + + # 🛡️ 音频裁剪与视频裁剪严格对齐：使用同一个 trim_val + ats = trim_val / fps if trim_val else 0.0 + if bus_audio_mode: + raw_wf = raw_audio["waveform"] if raw_audio else torch.zeros((1, 1)) + raw_sr = raw_audio["sample_rate"] if raw_audio else 44100 + # 对齐到 trim_val 帧：从当前累计位置开始，取 actual_out 帧长度的音频 + actual_out_frames = cut.shape[cd] + seg_start_sample = int((b["_accumulated_frames"] + trim_val) / fps * raw_sr) + seg_end_sample = int((b["_accumulated_frames"] + actual_out_frames + trim_val) / fps * raw_sr) + seg_end_sample = min(seg_end_sample, raw_wf.shape[-1]) + cut_audio = {"waveform": raw_wf[..., seg_start_sample:seg_end_sample], "sample_rate": raw_sr} if seg_end_sample > seg_start_sample else None + else: + cut_audio = raw_audio + if ats > 0 and raw_audio: + wf = raw_audio["waveform"] + sr = raw_audio["sample_rate"] + n = int(ats * sr) + if n < wf.shape[-1]: + cut_audio = {"waveform": wf[..., n:], "sample_rate": sr} + else: + cut_audio = None + + if accumulated_audio is None: + final_audio = cut_audio + elif cut_audio is None: + final_audio = accumulated_audio + else: + pa, ca = accumulated_audio, cut_audio + if pa["sample_rate"] != ca["sample_rate"]: + if torchaudio is None: + raise ImportError("🚨 [XB-BOX] 音频采样率不一致需要 torchaudio 重采样，但 torchaudio 未安装！请运行: pip install torchaudio") + ca_wf = torchaudio.functional.resample(ca["waveform"], ca["sample_rate"], pa["sample_rate"]) + ca = {"waveform": ca_wf, "sample_rate": pa["sample_rate"]} + pa_wf = pa["waveform"] + ca_wf = ca["waveform"].to(pa_wf.device) + if pa_wf.dim() >= 2 and ca_wf.dim() >= 2 and pa_wf.shape[:-1] != ca_wf.shape[:-1]: + if pa_wf.shape[-2] == 1 and ca_wf.shape[-2] > 1: + rpt = [1] * pa_wf.dim(); rpt[-2] = ca_wf.shape[-2]; pa_wf = pa_wf.repeat(*rpt) + elif ca_wf.shape[-2] == 1 and pa_wf.shape[-2] > 1: + rpt = [1] * ca_wf.dim(); rpt[-2] = pa_wf.shape[-2]; ca_wf = ca_wf.repeat(*rpt) + final_audio = {"waveform": torch.cat([pa_wf, ca_wf], dim=-1), "sample_rate": pa["sample_rate"]} + + b["_global_frame_offset"] = b.get("_global_frame_offset", 0) + cut.shape[cd] + if bus_audio_mode: + b["_accumulated_frames"] += cut.shape[cd] + accumulated_video = final_video + accumulated_audio = final_audio + print(f"✅ [XB-BOX] 接力 {r+1}/{relay_count} 完成. out={cut.shape[cd]}f (trim={trim_val}f) total={final_video.shape[cd]}f") + completed = r + 1 + + if completed < relay_count: + print(f"✅ [XB-BOX] 生成任务由 {completed} 个接力点已经完成，跳过剩余接力点！") + else: + print(f"🎉 [XB-BOX] InfiniteTalk 接力全部完成: {relay_count} 段, 总计 {accumulated_video.shape[0 if len(accumulated_video.shape)==4 else 1]} 帧") + # 👑 刷新总线全局状态：如果引入了独立参考图，将其升级为下游节点的新基准 + if is_local_ref: + b["start_image"] = local_start_image + b["clip_vision_output"] = local_clip_vision_output + print("🌐 [XB-BOX] 总线状态已更新：独立参考图已成为下游节点的新全局空间基准。") + + _refresh_models() + return (b, accumulated_video, accumulated_audio) diff --git a/nodes_save_video.py b/nodes/video/nodes_save_video.py similarity index 99% rename from nodes_save_video.py rename to nodes/video/nodes_save_video.py index c85414d..f030d08 100644 --- a/nodes_save_video.py +++ b/nodes/video/nodes_save_video.py @@ -18,7 +18,7 @@ from comfy.cli_args import args from comfy_api.latest import io, ui, InputImpl, Types from comfy_extras.nodes_video import CreateVideo, SaveVideo -from .xb_paths import resolve_filename_prefix +from ...libs.xb_paths import resolve_filename_prefix def _input_map(schema): diff --git a/nodes_video.py b/nodes/video/nodes_video.py similarity index 97% rename from nodes_video.py rename to nodes/video/nodes_video.py index b9488f7..f89c067 100644 --- a/nodes_video.py +++ b/nodes/video/nodes_video.py @@ -1,1153 +1,1153 @@ -import os -import itertools -import numpy as np -import torch -import cv2 -import hashlib -import re -import time -import subprocess -import shutil as _shutil - -import folder_paths -from comfy.utils import common_upscale, ProgressBar -import nodes - -from .xb_av import open_av -from .xb_paths import resolve_filename_prefix - -# ============================================================ -# 画幅比例 / 倍数 公共逻辑 -# 画幅比例 = 自定义 + 官方 Resolution Selector 的比例； -# 倍数（multiple）约束宽高：既作为调节步长，也保证宽高能整除。 -# ============================================================ -XB_ASPECT_RATIOS = [ - ("自定义", None), - ("1:1 (Square)", (1, 1)), - ("2:3 (Portrait Photo)", (2, 3)), - ("3:2 (Photo)", (3, 2)), - ("3:4 (Portrait Standard)", (3, 4)), - ("4:3 (Standard)", (4, 3)), - ("9:16 (Portrait Widescreen)", (9, 16)), - ("16:9 (Widescreen)", (16, 9)), - ("21:9 (Ultrawide)", (21, 9)), -] -XB_ASPECT_VALUES = [name for name, _ in XB_ASPECT_RATIOS] -XB_ASPECT_MAP = {name: r for name, r in XB_ASPECT_RATIOS if r} - -# 旧工作流里的值 → 新值（保证老图照常开） -XB_ASPECT_LEGACY = { - "Free": "自定义", - "1:1": "1:1 (Square)", - "2:3": "2:3 (Portrait Photo)", - "3:2": "3:2 (Photo)", - "3:4": "3:4 (Portrait Standard)", - "4:3": "4:3 (Standard)", - "9:16": "9:16 (Portrait Widescreen)", - "16:9": "16:9 (Widescreen)", - "21:9": "21:9 (Ultrawide)", -} - -XB_MULTIPLE_DEFAULT = 16 - - -def xb_multiple(value, default=XB_MULTIPLE_DEFAULT): - """把「倍数」规整成 >= 8 的整数。""" - try: - m = int(round(float(value))) - except Exception: - m = default - return max(8, m) - - -def xb_snap(value, m): - """吸附到 m 的整数倍（并保证 >= m）。""" - try: - v = int(round(float(value))) - except Exception: - v = m - return max(m, int(round(v / m)) * m) - - -def xb_aspect_ratio(name): - """画幅比例 → 宽高比（宽/高）；自定义 / 未知返回 None。""" - if name in XB_ASPECT_MAP: - w, h = XB_ASPECT_MAP[name] - return w / h - name = XB_ASPECT_LEGACY.get(name, name) - if name in XB_ASPECT_MAP: - w, h = XB_ASPECT_MAP[name] - return w / h - # 兼容 "16:9 (LTX)" 这类旧写法 - for key, ratio in XB_ASPECT_MAP.items(): - if name and name.startswith(key.split(" ")[0]): - w, h = ratio - return w / h - return None - - -def xb_fit_size(name, width, height, m): - """按比例 + 倍数算出最终宽高（两边都能被 m 整除）。""" - w = xb_snap(width, m) - h = xb_snap(height, m) - ratio = xb_aspect_ratio(name) - if ratio: - if w >= h: - h = xb_snap(w / ratio, m) - else: - w = xb_snap(h * ratio, m) - return max(m, w), max(m, h) - - -def xb_multiple_input(): - """「倍数」控件定义（放在最下面）。""" - return ("INT", { - "default": XB_MULTIPLE_DEFAULT, "min": 8, "max": 128, "step": 4, - "tooltip": "宽高必须是它的整数倍；调节分辨率时也按这个步长走（默认 16）。", - }) - -# ============================================================ -# XB_ImageParamsMaster — 图像参数主控 -# ============================================================ -class XB_ImageParamsMaster: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), - "width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), - "height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 1000, "step": 1}), - "strength_float": ("FLOAT", {"default": 1.00, "min": 0.00, "max": 10.00, "step": 0.01}), - "strength_int": ("INT", {"default": 1, "min": 0, "max": 1000, "step": 1}), - "multiple": xb_multiple_input(), - } - } - - RETURN_TYPES = ("INT", "INT", "INT", "FLOAT", "INT", "INT") - RETURN_NAMES = ("Image Width", "Image Height", "Batch Size", "Float Control", "Int Control", "Scale Size") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Image_Params" - - def process(self, aspect_ratio, width, height, batch_size, strength_float, strength_int, multiple=XB_MULTIPLE_DEFAULT): - m = xb_multiple(multiple) - safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) - return (safe_w, safe_h, batch_size, float(strength_float), int(strength_int), max(safe_w, safe_h)) - -# ============================================================ -# XB_ImageParamsMasterMini — 图像参数主控（精简版） -# 去掉「强度(小数)」「强度(整数)」两个控件与对应输出 -# 保留画幅比例联动 + 16 步长联动（JS 端负责） -# ============================================================ -class XB_ImageParamsMasterMini: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), - "width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), - "height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 1000, "step": 1}), - "multiple": xb_multiple_input(), - } - } - - RETURN_TYPES = ("INT", "INT", "INT", "INT") - RETURN_NAMES = ("Image Width", "Image Height", "Batch Size", "Scale Size") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Image_Params" - - def process(self, aspect_ratio, width, height, batch_size, multiple=XB_MULTIPLE_DEFAULT): - m = xb_multiple(multiple) - safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) - return (safe_w, safe_h, batch_size, max(safe_w, safe_h)) - -# ============================================================ -# XB_VideoParamsMaster — 视频参数主控 -# ============================================================ -class XB_VideoParamsMaster: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), - "duration_display": ("STRING", {"default": "Video Duration: 0.00 s", "multiline": False}), - "width": ("INT", {"default": 480, "min": 64, "max": 8192, "step": 16}), - "height": ("INT", {"default": 832, "min": 64, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 9999, "step": 1}), - "fps": ("INT", {"default": 16, "min": 1, "max": 120, "step": 1}), - "fps_float": ("FLOAT", {"default": 16.0, "min": 1.0, "max": 120.0, "step": 0.01}), - "multiple": xb_multiple_input(), - } - } - - RETURN_TYPES = ("INT", "INT", "INT", "INT", "FLOAT", "INT") - RETURN_NAMES = ("Width", "Height", "Frames", "FPS", "FPS_Float", "Scale Size") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Image_Params" - - def process(self, aspect_ratio, duration_display, width, height, length, fps, fps_float, - multiple=XB_MULTIPLE_DEFAULT): - final_fps = int(round(fps)) - m = xb_multiple(multiple) - safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) - - is_free = xb_aspect_ratio(aspect_ratio) is None - if is_free: - safe_len = max(1, int(length)) - else: - safe_len = max(1, round((length - 1) / 8) * 8 + 1) - - return (safe_w, safe_h, safe_len, final_fps, float(final_fps), max(safe_w, safe_h)) - -# ============================================================ -# XB_MasterParameter — 万能参数控制器 -# ============================================================ -class XB_MasterParameter: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "mode": (["Free Mode", "Model Mode", "Encode Mode", "Decode Mode", "Ratio Mode", "Other Mode"], {"default": "Free Mode"}), - "value": ("FLOAT", {"default": 1024.0, "min": 0.0, "max": 9999.0, "step": 0.01}), - } - } - - RETURN_TYPES = ("INT", "FLOAT") - RETURN_NAMES = ("Integer (INT)", "Float (FLOAT)") - FUNCTION = "get_value" - CATEGORY = "XB_ToolBox/Params_Control" - - def get_value(self, mode, value): - if mode == "Free Mode": - val = max(0.0, min(9999.0, value)) - elif mode == "Model Mode": - val = max(0.0, min(50.0, value)) - elif mode in ["Encode Mode", "Decode Mode"]: - val = max(64.0, min(3840.0, value)) - val = round(val / 32) * 32 - elif mode == "Ratio Mode": - val = max(0.0, min(1.0, value)) - elif mode == "Other Mode": - val = max(0.0, min(9999.0, value)) - else: - val = value - - return (int(round(val)), float(val)) - - -# ============================================================ -# 以下代码一字不差搬运自 VHS load_video_nodes.py 和 utils.py -# ============================================================ - -DIMMAX = 8192 -BIGMAX = (2**53-1) - -video_extensions = ['webm', 'mp4', 'mkv', 'gif', 'mov'] - -VHSLoadFormats = { - 'None': {}, - 'AnimateDiff': {'target_rate': 8, 'dim': (8,0,512,512)}, - 'Mochi': {'target_rate': 24, 'dim': (16,0,848,480), 'frames':(6,1)}, - 'LTXV': {'target_rate': 24, 'dim': (32,0,768,512), 'frames':(8,1)}, - 'Hunyuan': {'target_rate': 24, 'dim': (16,0,848,480), 'frames':(4,1)}, - 'Cosmos': {'target_rate': 24, 'dim': (16,0,1280,704), 'frames':(8,1)}, - 'Wan': {'target_rate': 16, 'dim': (8,0,832,480), 'frames':(4,1)}, -} - -if not hasattr(nodes, 'VHSLoadFormats'): - nodes.VHSLoadFormats = {} - -def get_load_formats(): - formats = {} - formats.update(nodes.VHSLoadFormats) - formats.update(VHSLoadFormats) - return (list(formats.keys()), - {'default': 'AnimateDiff', 'formats': formats}) - -def get_format(format): - if format in VHSLoadFormats: - return VHSLoadFormats[format] - return nodes.VHSLoadFormats.get(format, {}) - -def strip_path(path): - path = path.strip() - if path.startswith("\""): - path = path[1:] - if path.endswith("\""): - path = path[:-1] - return path - -def calculate_file_hash(filename: str, hash_every_n: int = 1): - h = hashlib.sha256() - h.update(filename.encode()) - h.update(str(os.path.getmtime(filename)).encode()) - return h.hexdigest() - -def target_size(width, height, custom_width, custom_height, downscale_ratio=8) -> tuple[int, int]: - if downscale_ratio is None: - downscale_ratio = 8 - if custom_width == 0 and custom_height == 0: - pass - elif custom_height == 0: - height *= custom_width/width - width = custom_width - elif custom_width == 0: - width *= custom_height/height - height = custom_height - else: - width = custom_width - height = custom_height - width = int(width/downscale_ratio + 0.5) * downscale_ratio - height = int(height/downscale_ratio + 0.5) * downscale_ratio - return (width, height) - -def cv_frame_generator(video, force_rate, frame_load_cap, skip_first_frames, - select_every_nth, meta_batch=None, unique_id=None): - video_cap = cv2.VideoCapture(video) - if not video_cap.isOpened() or not video_cap.grab(): - raise ValueError(f"{video} could not be loaded with cv.") - - # extract video metadata - fps = video_cap.get(cv2.CAP_PROP_FPS) - width = int(video_cap.get(cv2.CAP_PROP_FRAME_WIDTH)) - height = int(video_cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) - total_frames = int(video_cap.get(cv2.CAP_PROP_FRAME_COUNT)) - duration = total_frames / fps - - width = 0 - - if width <=0 or height <=0: - _, frame = video_cap.retrieve() - height, width, _ = frame.shape - - # set video_cap to look at start_index frame - total_frame_count = 0 - total_frames_evaluated = -1 - frames_added = 0 - base_frame_time = 1 / fps - prev_frame = None - - if force_rate == 0: - target_frame_time = base_frame_time - else: - target_frame_time = 1/force_rate - - if total_frames > 0: - if force_rate != 0: - yieldable_frames = int(total_frames / fps * force_rate) - else: - yieldable_frames = total_frames - if select_every_nth: - yieldable_frames //= select_every_nth - if frame_load_cap != 0: - yieldable_frames = min(frame_load_cap, yieldable_frames) - else: - yieldable_frames = 0 - yield (width, height, fps, duration, total_frames, target_frame_time, yieldable_frames) - pbar = ProgressBar(yieldable_frames) - time_offset=target_frame_time - while video_cap.isOpened(): - if time_offset < target_frame_time: - is_returned = video_cap.grab() - # if didn't return frame, video has ended - if not is_returned: - break - time_offset += base_frame_time - if time_offset < target_frame_time: - continue - time_offset -= target_frame_time - # if not at start_index, skip doing anything with frame - total_frame_count += 1 - if total_frame_count <= skip_first_frames: - continue - else: - total_frames_evaluated += 1 - - # if should not be selected, skip doing anything with frame - if total_frames_evaluated%select_every_nth != 0: - continue - - # opencv loads images in BGR format (yuck), so need to convert to RGB for ComfyUI use - # follow up: can videos ever have an alpha channel? - # To my testing: No. opencv has no support for alpha - unused, frame = video_cap.retrieve() - frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) - # convert frame to comfyui's expected format - # TODO: frame contains no exif information. Check if opencv2 has already applied - frame = np.array(frame, dtype=np.float32) - torch.from_numpy(frame).div_(255) - if prev_frame is not None: - inp = yield prev_frame - if inp is not None: - #ensure the finally block is called - return - prev_frame = frame - frames_added += 1 - if pbar is not None: - pbar.update_absolute(frames_added, yieldable_frames) - # if cap exists and we've reached it, stop processing frames - if frame_load_cap > 0 and frames_added >= frame_load_cap: - break - if meta_batch is not None: - meta_batch.inputs.pop(unique_id) - meta_batch.has_closed_inputs = True - if prev_frame is not None: - yield prev_frame - -#Python 3.12 adds an itertools.batched, but it's easily replicated for legacy support -def batched(it, n): - while batch := tuple(itertools.islice(it, n)): - yield batch - -def batched_vae_encode(images, vae, frames_per_batch): - for batch in batched(images, frames_per_batch): - image_batch = torch.from_numpy(np.array(batch)) - yield from vae.encode(image_batch).numpy() - -def resized_cv_frame_gen(custom_width, custom_height, downscale_ratio, **kwargs): - gen = cv_frame_generator(**kwargs) - info = next(gen) - width, height = info[0], info[1] - frames_per_batch = (1920 * 1080 * 16) // (width * height) or 1 - if kwargs.get('meta_batch', None) is not None: - frames_per_batch = min(frames_per_batch, kwargs['meta_batch'].frames_per_batch) - if custom_width != 0 or custom_height != 0 or downscale_ratio is not None: - new_size = target_size(width, height, custom_width, custom_height, downscale_ratio) - yield (*info, new_size[0], new_size[1], False) - if new_size[0] != width or new_size[1] != height: - def rescale(frame): - s = torch.from_numpy(np.fromiter(frame, np.dtype((np.float32, (height, width, 3))))) - s = s.movedim(-1,1) - s = common_upscale(s, new_size[0], new_size[1], "lanczos", "center") - return s.movedim(1,-1).numpy() - yield from itertools.chain.from_iterable(map(rescale, batched(gen, frames_per_batch))) - return - else: - yield (*info, info[0], info[1], False) - yield from gen - -def _lazy_get_audio(file, start_time=0, duration=0): - """替代 VHS 的 lazy_get_audio，使用 PyAV。视频无音轨时返回 None，不伪造静音。""" - try: - import av - except ImportError as exc: - raise RuntimeError("Video audio loading requires PyAV (av)") from exc - # 只探测流元数据，不解码；这样可选 AUDIO 消费端能拿到 None 而不是非法的单采样点张量 - try: - with open_av(file) as container: - if not any(stream.type == 'audio' for stream in container.streams): - return None - except Exception as exc: - raise RuntimeError("Unable to inspect video audio streams") from exc - class _LazyAudioMap: - def __init__(s, file, st, dur): - s._d = None; s.file = file; s.st = st; s.dur = dur - def _load(s): - if s._d is not None: return - try: - with open_av(s.file) as c: - a = next((x for x in c.streams if x.type == 'audio'), None) - if a is None: - raise ValueError("Audio stream disappeared after probing") - sr = a.codec_context.sample_rate or 44100 - frames = [] - sp = int(s.st*sr) if s.st>0 else 0 - ep = int((s.st+s.dur)*sr) if s.dur>0 else None - for f in c.decode(a): - if f.pts is not None: - if sp>0 and f.pts=ep: break - frames.append(f.to_ndarray()) - if not frames: - raise ValueError("Audio stream has no decoded frames in the requested interval") - wav = np.concatenate(frames, axis=-1) - if wav.size == 0: - raise ValueError("Audio stream contains no decoded samples") - if wav.ndim==1: wav=wav[np.newaxis,:] - elif wav.shape[0]>1: wav=wav.mean(axis=0, keepdims=True) - wf = torch.from_numpy(wav.astype(np.float32)) - if wf.dim()==2: wf=wf.unsqueeze(0) - s._d = {'waveform': wf, 'sample_rate': sr} - except Exception as exc: - raise RuntimeError("Unable to decode video audio") from exc - def __getitem__(s,k): s._load(); return s._d[k] - def __iter__(s): s._load(); return iter(s._d) - def __len__(s): s._load(); return len(s._d) - return _LazyAudioMap(file, start_time, duration) - -def load_video(meta_batch=None, unique_id=None, memory_limit_mb=None, vae=None, - generator=resized_cv_frame_gen, format='None', **kwargs): - if 'force_size' in kwargs: - kwargs.pop('force_size') - format = get_format(format) - kwargs['video'] = strip_path(kwargs['video']) - if vae is not None: - downscale_ratio = getattr(vae, "downscale_ratio", 8) - else: - downscale_ratio = format.get('dim', (1,))[0] - if meta_batch is None or unique_id not in meta_batch.inputs: - gen = generator(meta_batch=meta_batch, unique_id=unique_id, downscale_ratio=downscale_ratio, **kwargs) - (width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) = next(gen) - - if meta_batch is not None: - meta_batch.inputs[unique_id] = (gen, width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) - if yieldable_frames: - meta_batch.total_frames = min(meta_batch.total_frames, yieldable_frames) - else: - (gen, width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) = meta_batch.inputs[unique_id] - - memory_limit = None - if memory_limit_mb is not None: - memory_limit *= 2 ** 20 - else: - try: - import psutil - memory_limit = (psutil.virtual_memory().available + psutil.swap_memory().free) - 2 ** 27 - except: - memory_limit = BIGMAX - if vae is not None: - max_loadable_frames = int(memory_limit//(width*height*3*(4+4+1/10))) - else: - max_loadable_frames = int(memory_limit//(width*height*3*(.1))) - if meta_batch is not None: - if 'frames' in format: - if meta_batch.frames_per_batch % format['frames'][0] != format['frames'][1]: - error = (meta_batch.frames_per_batch - format['frames'][1]) % format['frames'][0] - suggested = meta_batch.frames_per_batch - error - if error > format['frames'][0] / 2: - suggested += format['frames'][0] - raise RuntimeError(f"The chosen frames per batch is incompatible with the selected format. Try {suggested}") - if meta_batch.frames_per_batch > max_loadable_frames: - raise RuntimeError(f"Meta Batch set to {meta_batch.frames_per_batch} frames but only {max_loadable_frames} can fit in memory") - gen = itertools.islice(gen, meta_batch.frames_per_batch) - else: - original_gen = gen - gen = itertools.islice(gen, max_loadable_frames) - frames_per_batch = (1920 * 1080 * 16) // (width * height) or 1 - if vae is not None: - gen = batched_vae_encode(gen, vae, frames_per_batch) - vw,vh = new_width//downscale_ratio, new_height//downscale_ratio - channels = getattr(vae, 'latent_channels', 4) - images = torch.from_numpy(np.fromiter(gen, np.dtype((np.float32, (channels,vh,vw))))) - else: - #Some minor wizardry to eliminate a copy and reduce max memory by a factor of ~2 - images = torch.from_numpy(np.fromiter(gen, np.dtype((np.float32, (new_height, new_width, 4 if alpha else 3))))) - if meta_batch is None and memory_limit is not None: - try: - next(original_gen) - raise RuntimeError(f"Memory limit hit after loading {len(images)} frames. Stopping execution.") - except StopIteration: - pass - if len(images) == 0: - raise RuntimeError("No frames generated") - if 'frames' in format and len(images) % format['frames'][0] != format['frames'][1]: - err_msg = f"The number of frames loaded {len(images)}, does not match the requirements of the currently selected format." - if len(format['frames']) > 2 and format['frames'][2]: - raise RuntimeError(err_msg) - div, mod = format['frames'][:2] - frames = (len(images) - mod) // div * div + mod - images = images[:frames] - if 'start_time' in kwargs: - start_time = kwargs['start_time'] - else: - start_time = kwargs['skip_first_frames'] * target_frame_time - target_frame_time *= kwargs.get('select_every_nth', 1) - #Setup lambda for lazy audio capture - audio = _lazy_get_audio(kwargs['video'], start_time, kwargs['frame_load_cap']*target_frame_time) - #Adjust target_frame_time for select_every_nth - video_info = { - "source_fps": fps, - "source_frame_count": total_frames, - "source_duration": duration, - "source_width": width, - "source_height": height, - "loaded_fps": 1/target_frame_time, - "loaded_frame_count": len(images), - "loaded_duration": len(images) * target_frame_time, - "loaded_width": new_width, - "loaded_height": new_height, - } - if vae is None: - return (images, len(images), audio, video_info) - else: - return ({"samples": images}, len(images), audio, video_info) - - -# ============================================================ -# XB_VideoLoader -# ============================================================ - -class XB_VideoLoader: - @classmethod - def INPUT_TYPES(cls): - input_dir = folder_paths.get_input_directory() - files = [] - for f in os.listdir(input_dir): - if os.path.isfile(os.path.join(input_dir, f)): - file_parts = f.split('.') - if len(file_parts) > 1 and (file_parts[-1].lower() in video_extensions): - files.append(f) - return {"required": { - "video": (sorted(files),), - "force_rate": ("FLOAT", {"default": 0, "min": 0, "max": 60, "step": 1}), - "custom_width": ("INT", {"default": 0, "min": 0, "max": DIMMAX}), - "custom_height": ("INT", {"default": 0, "min": 0, "max": DIMMAX}), - "frame_load_cap": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}), - "skip_first_frames": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}), - "select_every_nth": ("INT", {"default": 1, "min": 1, "max": BIGMAX, "step": 1}), - }, - "optional": { - "meta_batch": ("VHS_BatchManager",), - "vae": ("VAE",), - "format": get_load_formats(), - }, - "hidden": { - "unique_id": "UNIQUE_ID" - }, - } - - RETURN_TYPES = ("IMAGE", "INT", "AUDIO", "VHS_VIDEOINFO") - RETURN_NAMES = ("IMAGE", "frame_count", "audio", "video_info") - FUNCTION = "load_video" - CATEGORY = "XB_ToolBox/Video" - - def load_video(self, **kwargs): - kwargs['video'] = folder_paths.get_annotated_filepath(strip_path(kwargs['video'])) - return load_video(**kwargs) - - @classmethod - def IS_CHANGED(s, video, **kwargs): - image_path = folder_paths.get_annotated_filepath(video) - return calculate_file_hash(image_path) - - @classmethod - def VALIDATE_INPUTS(s, video): - if not folder_paths.exists_annotated_filepath(video): - return "Invalid video file: {}".format(video) - return True - - -# ============================================================ -# 以下代码搬运自 VHS utils.py 和 nodes.py -# ============================================================ - -# ffmpeg_path -if "VHS_FORCE_FFMPEG_PATH" in os.environ: - _ffmpeg = os.environ.get("VHS_FORCE_FFMPEG_PATH") -else: - _ffmpeg_paths = [] - try: - from imageio_ffmpeg import get_ffmpeg_exe - _ffmpeg_paths.append(get_ffmpeg_exe()) - except: - pass - system_ffmpeg = _shutil.which("ffmpeg") - if system_ffmpeg: - _ffmpeg_paths.append(system_ffmpeg) - if os.path.isfile("ffmpeg"): _ffmpeg_paths.append(os.path.abspath("ffmpeg")) - if os.path.isfile("ffmpeg.exe"): _ffmpeg_paths.append(os.path.abspath("ffmpeg.exe")) - _ffmpeg = _ffmpeg_paths[0] if _ffmpeg_paths else None - -import json as _json -import datetime as _dt -import sys as _sys -import functools as _functools -from PIL import Image as _PILImage, ExifTags as _ExifTags -from PIL.PngImagePlugin import PngInfo as _PngInfo -from string import Template as _Template - -# gifski_path -_gifski_path = os.environ.get("VHS_GIFSKI", None) -if _gifski_path is None: - _gifski_path = os.environ.get("JOV_GIFSKI", None) - if _gifski_path is None: - _gifski_path = _shutil.which("gifski") - -def _merge_filter_args(args, ftype="-vf"): - try: - start_index = args.index(ftype)+1 - index = start_index - while True: - index = args.index(ftype, index) - args[start_index] += ',' + args[index+1] - args.pop(index) - args.pop(index) - except ValueError: - pass - -def _cached(duration): - def dec(f): - cached_ret = None - cache_time = 0 - def cached_func(): - nonlocal cache_time, cached_ret - if time.time() > cache_time + duration or cached_ret is None: - cache_time = time.time() - cached_ret = f() - return cached_ret - return cached_func - return dec - -class _MultiInput(str): - def __new__(cls, string, allowed_types="*"): - res = super().__new__(cls, string) - res.allowed_types=allowed_types - return res - def __ne__(self, other): - if self.allowed_types == "*" or other == "*": - return False - return other not in self.allowed_types - -_imageOrLatent = _MultiInput("IMAGE", ["IMAGE", "LATENT"]) -_floatOrInt = _MultiInput("FLOAT", ["FLOAT", "INT"]) - -class _ContainsAll(dict): - def __contains__(self, other): - return True - def __getitem__(self, key): - return super().get(key, (None, {})) - -def _flatten_list(l): - ret = [] - for e in l: - if isinstance(e, list): - ret.extend(e) - else: - ret.append(e) - return ret - -def _iterate_format(video_format, for_widgets=True): - def indirector(cont, index): - if isinstance(cont[index], list) and (not for_widgets - or len(cont[index])> 1 and not isinstance(cont[index][1], dict)): - inp = yield cont[index] - if inp is not None: - cont[index] = inp - yield - for k in video_format: - if k == "extra_widgets": - if for_widgets: - yield from video_format["extra_widgets"] - elif k.endswith("_pass"): - for i in range(len(video_format[k])): - yield from indirector(video_format[k], i) - if not for_widgets: - video_format[k] = _flatten_list(video_format[k]) - else: - yield from indirector(video_format, k) - -if 'VHS_video_formats' not in folder_paths.folder_names_and_paths: - folder_paths.folder_names_and_paths["VHS_video_formats"] = ((),{".json"}) -if len(folder_paths.folder_names_and_paths['VHS_video_formats'][1]) == 0: - folder_paths.folder_names_and_paths["VHS_video_formats"][1].add(".json") - -_base_formats_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "video_formats") - -@_cached(5) -def _get_video_formats(): - format_files = {} - for format_name in folder_paths.get_filename_list("VHS_video_formats"): - format_files[format_name] = folder_paths.get_full_path("VHS_video_formats", format_name) - os.makedirs(_base_formats_dir, exist_ok=True) - for item in os.scandir(_base_formats_dir): - if not item.is_file() or not item.name.endswith('.json'): - continue - format_files[item.name[:-5]] = item.path - formats = [] - format_widgets = {} - for format_name, path in format_files.items(): - with open(path, 'r') as stream: - video_format = _json.load(stream) - if "gifski_pass" in video_format and _gifski_path is None: - continue - widgets = list(_iterate_format(video_format)) - formats.append("video/" + format_name) - if (len(widgets) > 0): - format_widgets["video/"+ format_name] = widgets - return formats, format_widgets - -def _apply_format_widgets(format_name, kwargs): - if os.path.exists(os.path.join(_base_formats_dir, format_name + ".json")): - video_format_path = os.path.join(_base_formats_dir, format_name + ".json") - else: - video_format_path = folder_paths.get_full_path("VHS_video_formats", format_name) - with open(video_format_path, 'r') as stream: - video_format = _json.load(stream) - for w in _iterate_format(video_format): - if w[0] not in kwargs: - if len(w) > 2 and 'default' in w[2]: - default = w[2]['default'] - else: - if type(w[1]) is list: - default = w[1][0] - else: - default = {"BOOLEAN": False, "INT": 0, "FLOAT": 0, "STRING": ""}[w[1]] - kwargs[w[0]] = default - wit = _iterate_format(video_format, False) - for w in wit: - while isinstance(w, list): - if len(w) == 1: - w = [_Template(x).substitute(**kwargs) for x in w[0]] - break - elif isinstance(w[1], dict): - w = w[1][str(kwargs[w[0]])] - elif len(w) > 3: - w = _Template(w[3]).substitute(val=kwargs[w[0]]) - else: - w = str(kwargs[w[0]]) - wit.send(w) - return video_format - -def _tensor_to_int(tensor, bits): - tensor = tensor.cpu().numpy() * (2**bits-1) + 0.5 - return np.clip(tensor, 0, (2**bits-1)) -def _tensor_to_shorts(tensor): - return _tensor_to_int(tensor, 16).astype(np.uint16) -def _tensor_to_bytes(tensor): - return _tensor_to_int(tensor, 8).astype(np.uint8) - -def _ffmpeg_process(args, video_format, video_metadata, file_path, env): - res = None - frame_data = yield - total_frames_output = 0 - if video_format.get('save_metadata', 'False') != 'False': - os.makedirs(folder_paths.get_temp_directory(), exist_ok=True) - metadata_path = os.path.join(folder_paths.get_temp_directory(), "metadata.txt") - def escape_ffmpeg_metadata(key, value): - value = str(value) - value = value.replace("\\","\\\\") - value = value.replace(";","\\;") - value = value.replace("#","\\#") - value = value.replace("=","\\=") - value = value.replace("\n","\\\n") - return f"{key}={value}" - with open(metadata_path, "w") as f: - f.write(";FFMETADATA1\n") - if "prompt" in video_metadata: - f.write(escape_ffmpeg_metadata("prompt", _json.dumps(video_metadata["prompt"])) + "\n") - if "workflow" in video_metadata: - f.write(escape_ffmpeg_metadata("workflow", _json.dumps(video_metadata["workflow"])) + "\n") - for k, v in video_metadata.items(): - if k not in ["prompt", "workflow"]: - f.write(escape_ffmpeg_metadata(k, _json.dumps(v)) + "\n") - m_args = args[:1] + ["-i", metadata_path] + args[1:] + ["-metadata", "creation_time=now", "-movflags", "use_metadata_tags"] - with subprocess.Popen(m_args + [file_path], stderr=subprocess.PIPE, - stdin=subprocess.PIPE, env=env) as proc: - try: - while frame_data is not None: - proc.stdin.write(frame_data) - frame_data = yield - total_frames_output+=1 - proc.stdin.flush() - proc.stdin.close() - res = proc.stderr.read() - except BrokenPipeError as e: - err = proc.stderr.read() - if os.path.exists(file_path): - raise Exception("An error occurred in the ffmpeg subprocess:\n" + err.decode(*ENCODE_ARGS)) - print(err.decode(*ENCODE_ARGS), end="", file=_sys.stderr) - if res != b'': - with subprocess.Popen(args + [file_path], stderr=subprocess.PIPE, - stdin=subprocess.PIPE, env=env) as proc: - try: - while frame_data is not None: - proc.stdin.write(frame_data) - frame_data = yield - total_frames_output+=1 - proc.stdin.flush() - proc.stdin.close() - res = proc.stderr.read() - except BrokenPipeError as e: - res = proc.stderr.read() - raise Exception("An error occurred in the ffmpeg subprocess:\n" + res.decode(*ENCODE_ARGS)) - yield total_frames_output - if len(res) > 0: - print(res.decode(*ENCODE_ARGS), end="", file=_sys.stderr) - -def _gifski_process(args, dimensions, frame_rate, video_format, file_path, env): - frame_data = yield - with subprocess.Popen(args + video_format['main_pass'] + ['-f', 'yuv4mpegpipe', '-'], - stderr=subprocess.PIPE, stdin=subprocess.PIPE, - stdout=subprocess.PIPE, env=env) as procff: - with subprocess.Popen([_gifski_path] + video_format['gifski_pass'] - + ['-W', f'{dimensions[0]}', '-H', f'{dimensions[1]}'] - + ['-r', f'{frame_rate}'] - + ['-q', '-o', file_path, '-'], stderr=subprocess.PIPE, - stdin=procff.stdout, stdout=subprocess.PIPE, - env=env) as procgs: - try: - while frame_data is not None: - procff.stdin.write(frame_data) - frame_data = yield - procff.stdin.flush() - procff.stdin.close() - resff = procff.stderr.read() - resgs = procgs.stderr.read() - outgs = procgs.stdout.read() - except BrokenPipeError as e: - procff.stdin.close() - resff = procff.stderr.read() - resgs = procgs.stderr.read() - raise Exception("An error occurred while creating gifski output\n" - + "Make sure you are using gifski --version >=1.32.0\nffmpeg: " - + resff.decode(*ENCODE_ARGS) + '\ngifski: ' + resgs.decode(*ENCODE_ARGS)) - if len(resff) > 0: - print(resff.decode(*ENCODE_ARGS), end="", file=_sys.stderr) - if len(resgs) > 0: - print(resgs.decode(*ENCODE_ARGS), end="", file=_sys.stderr) - if len(outgs) > 0: - print(outgs.decode(*ENCODE_ARGS)) - -def _to_pingpong(inp): - if not hasattr(inp, "__getitem__"): - inp = list(inp) - yield from inp - for i in range(len(inp)-2,0,-1): - yield inp[i] - - -# ============================================================ -# XB_VideoCombine — 完整搬运 VHS VideoCombine，完全独立 -# ============================================================ - -class XB_VideoCombine: - @classmethod - def INPUT_TYPES(cls): - ffmpeg_formats, format_widgets = _get_video_formats() - format_widgets["image/webp"] = [['lossless', "BOOLEAN", {'default': True}]] - return { - "required": { - "images": (_imageOrLatent,), - "frame_rate": (_floatOrInt, {"default": 8, "min": 1, "step": 1}), - "loop_count": ("INT", {"default": 0, "min": 0, "max": 100, "step": 1}), - "filename_prefix": ("STRING", {"default": "视频"}), - "format": (["image/gif", "image/webp"] + ffmpeg_formats, {'formats': format_widgets}), - "pingpong": ("BOOLEAN", {"default": False}), - "save_output": ("BOOLEAN", {"default": True}), - }, - "optional": { - "audio": ("AUDIO",), - }, - "hidden": _ContainsAll({ - "prompt": "PROMPT", - "extra_pnginfo": "EXTRA_PNGINFO", - "unique_id": "UNIQUE_ID" - }), - } - - RETURN_TYPES = ("VHS_FILENAMES",) - RETURN_NAMES = ("Filenames",) - OUTPUT_NODE = True - FUNCTION = "combine_video" - CATEGORY = "XB_ToolBox/Video" - - def combine_video(self, frame_rate, loop_count, images=None, latents=None, - filename_prefix="视频", format="image/gif", pingpong=False, - save_output=True, prompt=None, extra_pnginfo=None, - audio=None, unique_id=None, manual_format_widgets=None, **kwargs): - if latents is not None: - images = latents - if images is None: - return ((save_output, []),) - if isinstance(images, torch.Tensor) and images.size(0) == 0: - return ((save_output, []),) - num_frames = len(images) - pbar = ProgressBar(num_frames) - first_image = images[0] - while len(first_image.shape) > 3: - first_image = first_image[0] - images = iter(images) - - output_dir = folder_paths.get_output_directory() if save_output else folder_paths.get_temp_directory() - # 展开 %date:yyyy-MM-dd% 之类的标记，否则会生成非法目录名（Windows: WinError 267） - filename_prefix = resolve_filename_prefix(filename_prefix) - full_output_folder, filename, _, subfolder, _ = folder_paths.get_save_image_path(filename_prefix, output_dir) - output_files = [] - - metadata = _PngInfo() - video_metadata = {} - if prompt is not None: - metadata.add_text("prompt", _json.dumps(prompt)) - video_metadata["prompt"] = prompt - if extra_pnginfo is not None: - for x in extra_pnginfo: - metadata.add_text(x, _json.dumps(extra_pnginfo[x])) - video_metadata[x] = extra_pnginfo[x] - extra_options = extra_pnginfo.get('workflow', {}).get('extra', {}) - else: - extra_options = {} - metadata.add_text("CreationTime", _dt.datetime.now().isoformat(" ")[:19]) - - max_counter = 0 - matcher = re.compile(f"{re.escape(filename)}_(\\d+)\\D*\\..+", re.IGNORECASE) - for existing_file in os.listdir(full_output_folder): - match = matcher.fullmatch(existing_file) - if match: - max_counter = max(max_counter, int(match.group(1))) - counter = max_counter + 1 - - first_image_file = f"{filename}_{counter:05}.png" - file_path = os.path.join(full_output_folder, first_image_file) - if extra_options.get('VHS_MetadataImage', True) != False: - _PILImage.fromarray(_tensor_to_bytes(first_image)).save(file_path, pnginfo=metadata, compress_level=4) - output_files.append(file_path) - - format_type, format_ext = format.split("/") - if format_type == "image": - image_kwargs = {} - if format_ext == "gif": - image_kwargs['disposal'] = 2 - if format_ext == "webp": - exif = _PILImage.Exif() - exif[_ExifTags.IFD.Exif] = {36867: _dt.datetime.now().isoformat(" ")[:19]} - image_kwargs['exif'] = exif - image_kwargs['lossless'] = kwargs.get("lossless", True) - file = f"{filename}_{counter:05}.{format_ext}" - file_path = os.path.join(full_output_folder, file) - if pingpong: - images = _to_pingpong(images) - def frames_gen(images): - for i in images: - pbar.update(1) - yield _PILImage.fromarray(_tensor_to_bytes(i)) - frames = frames_gen(images) - next(frames).save(file_path, format=format_ext.upper(), save_all=True, - append_images=frames, duration=round(1000/frame_rate), - loop=loop_count, compress_level=4, **image_kwargs) - output_files.append(file_path) - else: - if _ffmpeg is None: - raise RuntimeError("ffmpeg 未找到") - - if manual_format_widgets is not None: - kwargs.update(manual_format_widgets) - - has_alpha = first_image.shape[-1] == 4 - kwargs["has_alpha"] = has_alpha - video_format = _apply_format_widgets(format_ext, kwargs) - dim_alignment = video_format.get("dim_alignment", 2) - if (first_image.shape[1] % dim_alignment) or (first_image.shape[0] % dim_alignment): - to_pad = (-first_image.shape[1] % dim_alignment, -first_image.shape[0] % dim_alignment) - padding = (to_pad[0]//2, to_pad[0] - to_pad[0]//2, to_pad[1]//2, to_pad[1] - to_pad[1]//2) - padfunc = torch.nn.ReplicationPad2d(padding) - def pad(image): - image = image.permute((2,0,1)) - padded = padfunc(image.to(dtype=torch.float32)) - return padded.permute((1,2,0)) - images = map(pad, images) - dimensions = (-first_image.shape[1] % dim_alignment + first_image.shape[1], - -first_image.shape[0] % dim_alignment + first_image.shape[0]) - else: - dimensions = (first_image.shape[1], first_image.shape[0]) - if pingpong: - images = _to_pingpong(images) - if num_frames > 2: - num_frames += num_frames -2 - pbar.total = num_frames - if loop_count > 0: - loop_args = ["-vf", "loop=loop=" + str(loop_count)+":size=" + str(num_frames)] - else: - loop_args = [] - if video_format.get('input_color_depth', '8bit') == '16bit': - images = map(_tensor_to_shorts, images) - i_pix_fmt = 'rgba64' if has_alpha else 'rgb48' - else: - images = map(_tensor_to_bytes, images) - i_pix_fmt = 'rgba' if has_alpha else 'rgb24' - file = f"{filename}_{counter:05}.{video_format['extension']}" - file_path = os.path.join(full_output_folder, file) - bitrate_arg = [] - bitrate = video_format.get('bitrate') - if bitrate is not None: - bitrate_arg = ["-b:v", str(bitrate) + "M" if video_format.get('megabit') == 'True' else str(bitrate) + "K"] - args = [_ffmpeg, "-v", "error", "-f", "rawvideo", "-pix_fmt", i_pix_fmt, - "-color_range", "pc", "-colorspace", "rgb", "-color_primaries", "bt709", - "-color_trc", video_format.get("fake_trc", "iec61966-2-1"), - "-s", f"{dimensions[0]}x{dimensions[1]}", "-r", str(frame_rate), "-i", "-"] + loop_args - - images = map(lambda x: x.tobytes(), images) - env = os.environ.copy() - if "environment" in video_format: - env.update(video_format["environment"]) - - if "pre_pass" in video_format: - images = [b''.join(images)] - os.makedirs(folder_paths.get_temp_directory(), exist_ok=True) - in_args_len = args.index("-i") + 2 - pre_pass_args = args[:in_args_len] + video_format['pre_pass'] - _merge_filter_args(pre_pass_args) - try: - subprocess.run(pre_pass_args, input=images[0], env=env, capture_output=True, check=True) - except subprocess.CalledProcessError as e: - raise Exception("An error occurred in the ffmpeg prepass:\n" + e.stderr.decode(*ENCODE_ARGS)) - if "inputs_main_pass" in video_format: - in_args_len = args.index("-i") + 2 - args = args[:in_args_len] + video_format['inputs_main_pass'] + args[in_args_len:] - - if 'gifski_pass' in video_format: - format = 'image/gif' - output_process = _gifski_process(args, dimensions, frame_rate, video_format, file_path, env) - audio = None - else: - args += video_format['main_pass'] + bitrate_arg - _merge_filter_args(args) - output_process = _ffmpeg_process(args, video_format, video_metadata, file_path, env) - output_process.send(None) - - for image in images: - pbar.update(1) - output_process.send(image) - try: - total_frames_output = output_process.send(None) - output_process.send(None) - except StopIteration: - pass - output_files.append(file_path) - - a_waveform = None - if audio is not None: - try: - a_waveform = audio['waveform'] - except: - pass - if a_waveform is not None: - output_file_with_audio = f"{filename}_{counter:05}-audio.{video_format['extension']}" - output_file_with_audio_path = os.path.join(full_output_folder, output_file_with_audio) - if "audio_pass" not in video_format: - video_format["audio_pass"] = ["-c:a", "libopus"] - channels = audio['waveform'].size(1) - min_audio_dur = total_frames_output / frame_rate + 1 - if video_format.get('trim_to_audio', 'False') != 'False': - apad = [] - else: - apad = ["-af", "apad=whole_dur="+str(min_audio_dur)] - mux_args = [_ffmpeg, "-v", "error", "-n", "-i", file_path, - "-ar", str(audio['sample_rate']), "-ac", str(channels), - "-f", "f32le", "-i", "-", "-c:v", "copy"] \ - + video_format["audio_pass"] + apad + ["-shortest", output_file_with_audio_path] - audio_data = audio['waveform'].squeeze(0).transpose(0,1).numpy().tobytes() - _merge_filter_args(mux_args, '-af') - try: - res = subprocess.run(mux_args, input=audio_data, env=env, capture_output=True, check=True) - except subprocess.CalledProcessError as e: - raise Exception("An error occured in the ffmpeg subprocess:\n" + e.stderr.decode(*ENCODE_ARGS)) - output_files.append(output_file_with_audio_path) - file = output_file_with_audio - if extra_options.get('VHS_KeepIntermediate', True) == False: - for intermediate in output_files[1:-1]: - if os.path.exists(intermediate): - os.remove(intermediate) - preview = { - "filename": file, - "subfolder": subfolder, - "type": "output" if save_output else "temp", - "format": format, - "frame_rate": frame_rate, - "workflow": first_image_file, - "fullpath": output_files[-1], - } - if num_frames == 1 and 'png' in format and '%03d' in file: - preview['format'] = 'image/png' - preview['filename'] = file.replace('%03d', '001') +import os +import itertools +import numpy as np +import torch +import cv2 +import hashlib +import re +import time +import subprocess +import shutil as _shutil + +import folder_paths +from comfy.utils import common_upscale, ProgressBar +import nodes + +from ...libs.xb_av import open_av +from ...libs.xb_paths import resolve_filename_prefix + +# ============================================================ +# 画幅比例 / 倍数 公共逻辑 +# 画幅比例 = 自定义 + 官方 Resolution Selector 的比例； +# 倍数（multiple）约束宽高：既作为调节步长，也保证宽高能整除。 +# ============================================================ +XB_ASPECT_RATIOS = [ + ("自定义", None), + ("1:1 (Square)", (1, 1)), + ("2:3 (Portrait Photo)", (2, 3)), + ("3:2 (Photo)", (3, 2)), + ("3:4 (Portrait Standard)", (3, 4)), + ("4:3 (Standard)", (4, 3)), + ("9:16 (Portrait Widescreen)", (9, 16)), + ("16:9 (Widescreen)", (16, 9)), + ("21:9 (Ultrawide)", (21, 9)), +] +XB_ASPECT_VALUES = [name for name, _ in XB_ASPECT_RATIOS] +XB_ASPECT_MAP = {name: r for name, r in XB_ASPECT_RATIOS if r} + +# 旧工作流里的值 → 新值（保证老图照常开） +XB_ASPECT_LEGACY = { + "Free": "自定义", + "1:1": "1:1 (Square)", + "2:3": "2:3 (Portrait Photo)", + "3:2": "3:2 (Photo)", + "3:4": "3:4 (Portrait Standard)", + "4:3": "4:3 (Standard)", + "9:16": "9:16 (Portrait Widescreen)", + "16:9": "16:9 (Widescreen)", + "21:9": "21:9 (Ultrawide)", +} + +XB_MULTIPLE_DEFAULT = 16 + + +def xb_multiple(value, default=XB_MULTIPLE_DEFAULT): + """把「倍数」规整成 >= 8 的整数。""" + try: + m = int(round(float(value))) + except Exception: + m = default + return max(8, m) + + +def xb_snap(value, m): + """吸附到 m 的整数倍（并保证 >= m）。""" + try: + v = int(round(float(value))) + except Exception: + v = m + return max(m, int(round(v / m)) * m) + + +def xb_aspect_ratio(name): + """画幅比例 → 宽高比（宽/高）；自定义 / 未知返回 None。""" + if name in XB_ASPECT_MAP: + w, h = XB_ASPECT_MAP[name] + return w / h + name = XB_ASPECT_LEGACY.get(name, name) + if name in XB_ASPECT_MAP: + w, h = XB_ASPECT_MAP[name] + return w / h + # 兼容 "16:9 (LTX)" 这类旧写法 + for key, ratio in XB_ASPECT_MAP.items(): + if name and name.startswith(key.split(" ")[0]): + w, h = ratio + return w / h + return None + + +def xb_fit_size(name, width, height, m): + """按比例 + 倍数算出最终宽高（两边都能被 m 整除）。""" + w = xb_snap(width, m) + h = xb_snap(height, m) + ratio = xb_aspect_ratio(name) + if ratio: + if w >= h: + h = xb_snap(w / ratio, m) + else: + w = xb_snap(h * ratio, m) + return max(m, w), max(m, h) + + +def xb_multiple_input(): + """「倍数」控件定义（放在最下面）。""" + return ("INT", { + "default": XB_MULTIPLE_DEFAULT, "min": 8, "max": 128, "step": 4, + "tooltip": "宽高必须是它的整数倍；调节分辨率时也按这个步长走（默认 16）。", + }) + +# ============================================================ +# XB_ImageParamsMaster — 图像参数主控 +# ============================================================ +class XB_ImageParamsMaster: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), + "width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), + "height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 1000, "step": 1}), + "strength_float": ("FLOAT", {"default": 1.00, "min": 0.00, "max": 10.00, "step": 0.01}), + "strength_int": ("INT", {"default": 1, "min": 0, "max": 1000, "step": 1}), + "multiple": xb_multiple_input(), + } + } + + RETURN_TYPES = ("INT", "INT", "INT", "FLOAT", "INT", "INT") + RETURN_NAMES = ("Image Width", "Image Height", "Batch Size", "Float Control", "Int Control", "Scale Size") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Image_Params" + + def process(self, aspect_ratio, width, height, batch_size, strength_float, strength_int, multiple=XB_MULTIPLE_DEFAULT): + m = xb_multiple(multiple) + safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) + return (safe_w, safe_h, batch_size, float(strength_float), int(strength_int), max(safe_w, safe_h)) + +# ============================================================ +# XB_ImageParamsMasterMini — 图像参数主控（精简版） +# 去掉「强度(小数)」「强度(整数)」两个控件与对应输出 +# 保留画幅比例联动 + 16 步长联动（JS 端负责） +# ============================================================ +class XB_ImageParamsMasterMini: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), + "width": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), + "height": ("INT", {"default": 1024, "min": 64, "max": 8192, "step": 16}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 1000, "step": 1}), + "multiple": xb_multiple_input(), + } + } + + RETURN_TYPES = ("INT", "INT", "INT", "INT") + RETURN_NAMES = ("Image Width", "Image Height", "Batch Size", "Scale Size") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Image_Params" + + def process(self, aspect_ratio, width, height, batch_size, multiple=XB_MULTIPLE_DEFAULT): + m = xb_multiple(multiple) + safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) + return (safe_w, safe_h, batch_size, max(safe_w, safe_h)) + +# ============================================================ +# XB_VideoParamsMaster — 视频参数主控 +# ============================================================ +class XB_VideoParamsMaster: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "aspect_ratio": (XB_ASPECT_VALUES, {"default": "自定义"}), + "duration_display": ("STRING", {"default": "Video Duration: 0.00 s", "multiline": False}), + "width": ("INT", {"default": 480, "min": 64, "max": 8192, "step": 16}), + "height": ("INT", {"default": 832, "min": 64, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 9999, "step": 1}), + "fps": ("INT", {"default": 16, "min": 1, "max": 120, "step": 1}), + "fps_float": ("FLOAT", {"default": 16.0, "min": 1.0, "max": 120.0, "step": 0.01}), + "multiple": xb_multiple_input(), + } + } + + RETURN_TYPES = ("INT", "INT", "INT", "INT", "FLOAT", "INT") + RETURN_NAMES = ("Width", "Height", "Frames", "FPS", "FPS_Float", "Scale Size") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Image_Params" + + def process(self, aspect_ratio, duration_display, width, height, length, fps, fps_float, + multiple=XB_MULTIPLE_DEFAULT): + final_fps = int(round(fps)) + m = xb_multiple(multiple) + safe_w, safe_h = xb_fit_size(aspect_ratio, width, height, m) + + is_free = xb_aspect_ratio(aspect_ratio) is None + if is_free: + safe_len = max(1, int(length)) + else: + safe_len = max(1, round((length - 1) / 8) * 8 + 1) + + return (safe_w, safe_h, safe_len, final_fps, float(final_fps), max(safe_w, safe_h)) + +# ============================================================ +# XB_MasterParameter — 万能参数控制器 +# ============================================================ +class XB_MasterParameter: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "mode": (["Free Mode", "Model Mode", "Encode Mode", "Decode Mode", "Ratio Mode", "Other Mode"], {"default": "Free Mode"}), + "value": ("FLOAT", {"default": 1024.0, "min": 0.0, "max": 9999.0, "step": 0.01}), + } + } + + RETURN_TYPES = ("INT", "FLOAT") + RETURN_NAMES = ("Integer (INT)", "Float (FLOAT)") + FUNCTION = "get_value" + CATEGORY = "XB_ToolBox/Params_Control" + + def get_value(self, mode, value): + if mode == "Free Mode": + val = max(0.0, min(9999.0, value)) + elif mode == "Model Mode": + val = max(0.0, min(50.0, value)) + elif mode in ["Encode Mode", "Decode Mode"]: + val = max(64.0, min(3840.0, value)) + val = round(val / 32) * 32 + elif mode == "Ratio Mode": + val = max(0.0, min(1.0, value)) + elif mode == "Other Mode": + val = max(0.0, min(9999.0, value)) + else: + val = value + + return (int(round(val)), float(val)) + + +# ============================================================ +# 以下代码一字不差搬运自 VHS load_video_nodes.py 和 utils.py +# ============================================================ + +DIMMAX = 8192 +BIGMAX = (2**53-1) + +video_extensions = ['webm', 'mp4', 'mkv', 'gif', 'mov'] + +VHSLoadFormats = { + 'None': {}, + 'AnimateDiff': {'target_rate': 8, 'dim': (8,0,512,512)}, + 'Mochi': {'target_rate': 24, 'dim': (16,0,848,480), 'frames':(6,1)}, + 'LTXV': {'target_rate': 24, 'dim': (32,0,768,512), 'frames':(8,1)}, + 'Hunyuan': {'target_rate': 24, 'dim': (16,0,848,480), 'frames':(4,1)}, + 'Cosmos': {'target_rate': 24, 'dim': (16,0,1280,704), 'frames':(8,1)}, + 'Wan': {'target_rate': 16, 'dim': (8,0,832,480), 'frames':(4,1)}, +} + +if not hasattr(nodes, 'VHSLoadFormats'): + nodes.VHSLoadFormats = {} + +def get_load_formats(): + formats = {} + formats.update(nodes.VHSLoadFormats) + formats.update(VHSLoadFormats) + return (list(formats.keys()), + {'default': 'AnimateDiff', 'formats': formats}) + +def get_format(format): + if format in VHSLoadFormats: + return VHSLoadFormats[format] + return nodes.VHSLoadFormats.get(format, {}) + +def strip_path(path): + path = path.strip() + if path.startswith("\""): + path = path[1:] + if path.endswith("\""): + path = path[:-1] + return path + +def calculate_file_hash(filename: str, hash_every_n: int = 1): + h = hashlib.sha256() + h.update(filename.encode()) + h.update(str(os.path.getmtime(filename)).encode()) + return h.hexdigest() + +def target_size(width, height, custom_width, custom_height, downscale_ratio=8) -> tuple[int, int]: + if downscale_ratio is None: + downscale_ratio = 8 + if custom_width == 0 and custom_height == 0: + pass + elif custom_height == 0: + height *= custom_width/width + width = custom_width + elif custom_width == 0: + width *= custom_height/height + height = custom_height + else: + width = custom_width + height = custom_height + width = int(width/downscale_ratio + 0.5) * downscale_ratio + height = int(height/downscale_ratio + 0.5) * downscale_ratio + return (width, height) + +def cv_frame_generator(video, force_rate, frame_load_cap, skip_first_frames, + select_every_nth, meta_batch=None, unique_id=None): + video_cap = cv2.VideoCapture(video) + if not video_cap.isOpened() or not video_cap.grab(): + raise ValueError(f"{video} could not be loaded with cv.") + + # extract video metadata + fps = video_cap.get(cv2.CAP_PROP_FPS) + width = int(video_cap.get(cv2.CAP_PROP_FRAME_WIDTH)) + height = int(video_cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) + total_frames = int(video_cap.get(cv2.CAP_PROP_FRAME_COUNT)) + duration = total_frames / fps + + width = 0 + + if width <=0 or height <=0: + _, frame = video_cap.retrieve() + height, width, _ = frame.shape + + # set video_cap to look at start_index frame + total_frame_count = 0 + total_frames_evaluated = -1 + frames_added = 0 + base_frame_time = 1 / fps + prev_frame = None + + if force_rate == 0: + target_frame_time = base_frame_time + else: + target_frame_time = 1/force_rate + + if total_frames > 0: + if force_rate != 0: + yieldable_frames = int(total_frames / fps * force_rate) + else: + yieldable_frames = total_frames + if select_every_nth: + yieldable_frames //= select_every_nth + if frame_load_cap != 0: + yieldable_frames = min(frame_load_cap, yieldable_frames) + else: + yieldable_frames = 0 + yield (width, height, fps, duration, total_frames, target_frame_time, yieldable_frames) + pbar = ProgressBar(yieldable_frames) + time_offset=target_frame_time + while video_cap.isOpened(): + if time_offset < target_frame_time: + is_returned = video_cap.grab() + # if didn't return frame, video has ended + if not is_returned: + break + time_offset += base_frame_time + if time_offset < target_frame_time: + continue + time_offset -= target_frame_time + # if not at start_index, skip doing anything with frame + total_frame_count += 1 + if total_frame_count <= skip_first_frames: + continue + else: + total_frames_evaluated += 1 + + # if should not be selected, skip doing anything with frame + if total_frames_evaluated%select_every_nth != 0: + continue + + # opencv loads images in BGR format (yuck), so need to convert to RGB for ComfyUI use + # follow up: can videos ever have an alpha channel? + # To my testing: No. opencv has no support for alpha + unused, frame = video_cap.retrieve() + frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) + # convert frame to comfyui's expected format + # TODO: frame contains no exif information. Check if opencv2 has already applied + frame = np.array(frame, dtype=np.float32) + torch.from_numpy(frame).div_(255) + if prev_frame is not None: + inp = yield prev_frame + if inp is not None: + #ensure the finally block is called + return + prev_frame = frame + frames_added += 1 + if pbar is not None: + pbar.update_absolute(frames_added, yieldable_frames) + # if cap exists and we've reached it, stop processing frames + if frame_load_cap > 0 and frames_added >= frame_load_cap: + break + if meta_batch is not None: + meta_batch.inputs.pop(unique_id) + meta_batch.has_closed_inputs = True + if prev_frame is not None: + yield prev_frame + +#Python 3.12 adds an itertools.batched, but it's easily replicated for legacy support +def batched(it, n): + while batch := tuple(itertools.islice(it, n)): + yield batch + +def batched_vae_encode(images, vae, frames_per_batch): + for batch in batched(images, frames_per_batch): + image_batch = torch.from_numpy(np.array(batch)) + yield from vae.encode(image_batch).numpy() + +def resized_cv_frame_gen(custom_width, custom_height, downscale_ratio, **kwargs): + gen = cv_frame_generator(**kwargs) + info = next(gen) + width, height = info[0], info[1] + frames_per_batch = (1920 * 1080 * 16) // (width * height) or 1 + if kwargs.get('meta_batch', None) is not None: + frames_per_batch = min(frames_per_batch, kwargs['meta_batch'].frames_per_batch) + if custom_width != 0 or custom_height != 0 or downscale_ratio is not None: + new_size = target_size(width, height, custom_width, custom_height, downscale_ratio) + yield (*info, new_size[0], new_size[1], False) + if new_size[0] != width or new_size[1] != height: + def rescale(frame): + s = torch.from_numpy(np.fromiter(frame, np.dtype((np.float32, (height, width, 3))))) + s = s.movedim(-1,1) + s = common_upscale(s, new_size[0], new_size[1], "lanczos", "center") + return s.movedim(1,-1).numpy() + yield from itertools.chain.from_iterable(map(rescale, batched(gen, frames_per_batch))) + return + else: + yield (*info, info[0], info[1], False) + yield from gen + +def _lazy_get_audio(file, start_time=0, duration=0): + """替代 VHS 的 lazy_get_audio，使用 PyAV。视频无音轨时返回 None，不伪造静音。""" + try: + import av + except ImportError as exc: + raise RuntimeError("Video audio loading requires PyAV (av)") from exc + # 只探测流元数据，不解码；这样可选 AUDIO 消费端能拿到 None 而不是非法的单采样点张量 + try: + with open_av(file) as container: + if not any(stream.type == 'audio' for stream in container.streams): + return None + except Exception as exc: + raise RuntimeError("Unable to inspect video audio streams") from exc + class _LazyAudioMap: + def __init__(s, file, st, dur): + s._d = None; s.file = file; s.st = st; s.dur = dur + def _load(s): + if s._d is not None: return + try: + with open_av(s.file) as c: + a = next((x for x in c.streams if x.type == 'audio'), None) + if a is None: + raise ValueError("Audio stream disappeared after probing") + sr = a.codec_context.sample_rate or 44100 + frames = [] + sp = int(s.st*sr) if s.st>0 else 0 + ep = int((s.st+s.dur)*sr) if s.dur>0 else None + for f in c.decode(a): + if f.pts is not None: + if sp>0 and f.pts=ep: break + frames.append(f.to_ndarray()) + if not frames: + raise ValueError("Audio stream has no decoded frames in the requested interval") + wav = np.concatenate(frames, axis=-1) + if wav.size == 0: + raise ValueError("Audio stream contains no decoded samples") + if wav.ndim==1: wav=wav[np.newaxis,:] + elif wav.shape[0]>1: wav=wav.mean(axis=0, keepdims=True) + wf = torch.from_numpy(wav.astype(np.float32)) + if wf.dim()==2: wf=wf.unsqueeze(0) + s._d = {'waveform': wf, 'sample_rate': sr} + except Exception as exc: + raise RuntimeError("Unable to decode video audio") from exc + def __getitem__(s,k): s._load(); return s._d[k] + def __iter__(s): s._load(); return iter(s._d) + def __len__(s): s._load(); return len(s._d) + return _LazyAudioMap(file, start_time, duration) + +def load_video(meta_batch=None, unique_id=None, memory_limit_mb=None, vae=None, + generator=resized_cv_frame_gen, format='None', **kwargs): + if 'force_size' in kwargs: + kwargs.pop('force_size') + format = get_format(format) + kwargs['video'] = strip_path(kwargs['video']) + if vae is not None: + downscale_ratio = getattr(vae, "downscale_ratio", 8) + else: + downscale_ratio = format.get('dim', (1,))[0] + if meta_batch is None or unique_id not in meta_batch.inputs: + gen = generator(meta_batch=meta_batch, unique_id=unique_id, downscale_ratio=downscale_ratio, **kwargs) + (width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) = next(gen) + + if meta_batch is not None: + meta_batch.inputs[unique_id] = (gen, width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) + if yieldable_frames: + meta_batch.total_frames = min(meta_batch.total_frames, yieldable_frames) + else: + (gen, width, height, fps, duration, total_frames, target_frame_time, yieldable_frames, new_width, new_height, alpha) = meta_batch.inputs[unique_id] + + memory_limit = None + if memory_limit_mb is not None: + memory_limit *= 2 ** 20 + else: + try: + import psutil + memory_limit = (psutil.virtual_memory().available + psutil.swap_memory().free) - 2 ** 27 + except: + memory_limit = BIGMAX + if vae is not None: + max_loadable_frames = int(memory_limit//(width*height*3*(4+4+1/10))) + else: + max_loadable_frames = int(memory_limit//(width*height*3*(.1))) + if meta_batch is not None: + if 'frames' in format: + if meta_batch.frames_per_batch % format['frames'][0] != format['frames'][1]: + error = (meta_batch.frames_per_batch - format['frames'][1]) % format['frames'][0] + suggested = meta_batch.frames_per_batch - error + if error > format['frames'][0] / 2: + suggested += format['frames'][0] + raise RuntimeError(f"The chosen frames per batch is incompatible with the selected format. Try {suggested}") + if meta_batch.frames_per_batch > max_loadable_frames: + raise RuntimeError(f"Meta Batch set to {meta_batch.frames_per_batch} frames but only {max_loadable_frames} can fit in memory") + gen = itertools.islice(gen, meta_batch.frames_per_batch) + else: + original_gen = gen + gen = itertools.islice(gen, max_loadable_frames) + frames_per_batch = (1920 * 1080 * 16) // (width * height) or 1 + if vae is not None: + gen = batched_vae_encode(gen, vae, frames_per_batch) + vw,vh = new_width//downscale_ratio, new_height//downscale_ratio + channels = getattr(vae, 'latent_channels', 4) + images = torch.from_numpy(np.fromiter(gen, np.dtype((np.float32, (channels,vh,vw))))) + else: + #Some minor wizardry to eliminate a copy and reduce max memory by a factor of ~2 + images = torch.from_numpy(np.fromiter(gen, np.dtype((np.float32, (new_height, new_width, 4 if alpha else 3))))) + if meta_batch is None and memory_limit is not None: + try: + next(original_gen) + raise RuntimeError(f"Memory limit hit after loading {len(images)} frames. Stopping execution.") + except StopIteration: + pass + if len(images) == 0: + raise RuntimeError("No frames generated") + if 'frames' in format and len(images) % format['frames'][0] != format['frames'][1]: + err_msg = f"The number of frames loaded {len(images)}, does not match the requirements of the currently selected format." + if len(format['frames']) > 2 and format['frames'][2]: + raise RuntimeError(err_msg) + div, mod = format['frames'][:2] + frames = (len(images) - mod) // div * div + mod + images = images[:frames] + if 'start_time' in kwargs: + start_time = kwargs['start_time'] + else: + start_time = kwargs['skip_first_frames'] * target_frame_time + target_frame_time *= kwargs.get('select_every_nth', 1) + #Setup lambda for lazy audio capture + audio = _lazy_get_audio(kwargs['video'], start_time, kwargs['frame_load_cap']*target_frame_time) + #Adjust target_frame_time for select_every_nth + video_info = { + "source_fps": fps, + "source_frame_count": total_frames, + "source_duration": duration, + "source_width": width, + "source_height": height, + "loaded_fps": 1/target_frame_time, + "loaded_frame_count": len(images), + "loaded_duration": len(images) * target_frame_time, + "loaded_width": new_width, + "loaded_height": new_height, + } + if vae is None: + return (images, len(images), audio, video_info) + else: + return ({"samples": images}, len(images), audio, video_info) + + +# ============================================================ +# XB_VideoLoader +# ============================================================ + +class XB_VideoLoader: + @classmethod + def INPUT_TYPES(cls): + input_dir = folder_paths.get_input_directory() + files = [] + for f in os.listdir(input_dir): + if os.path.isfile(os.path.join(input_dir, f)): + file_parts = f.split('.') + if len(file_parts) > 1 and (file_parts[-1].lower() in video_extensions): + files.append(f) + return {"required": { + "video": (sorted(files),), + "force_rate": ("FLOAT", {"default": 0, "min": 0, "max": 60, "step": 1}), + "custom_width": ("INT", {"default": 0, "min": 0, "max": DIMMAX}), + "custom_height": ("INT", {"default": 0, "min": 0, "max": DIMMAX}), + "frame_load_cap": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}), + "skip_first_frames": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}), + "select_every_nth": ("INT", {"default": 1, "min": 1, "max": BIGMAX, "step": 1}), + }, + "optional": { + "meta_batch": ("VHS_BatchManager",), + "vae": ("VAE",), + "format": get_load_formats(), + }, + "hidden": { + "unique_id": "UNIQUE_ID" + }, + } + + RETURN_TYPES = ("IMAGE", "INT", "AUDIO", "VHS_VIDEOINFO") + RETURN_NAMES = ("IMAGE", "frame_count", "audio", "video_info") + FUNCTION = "load_video" + CATEGORY = "XB_ToolBox/Video" + + def load_video(self, **kwargs): + kwargs['video'] = folder_paths.get_annotated_filepath(strip_path(kwargs['video'])) + return load_video(**kwargs) + + @classmethod + def IS_CHANGED(s, video, **kwargs): + image_path = folder_paths.get_annotated_filepath(video) + return calculate_file_hash(image_path) + + @classmethod + def VALIDATE_INPUTS(s, video): + if not folder_paths.exists_annotated_filepath(video): + return "Invalid video file: {}".format(video) + return True + + +# ============================================================ +# 以下代码搬运自 VHS utils.py 和 nodes.py +# ============================================================ + +# ffmpeg_path +if "VHS_FORCE_FFMPEG_PATH" in os.environ: + _ffmpeg = os.environ.get("VHS_FORCE_FFMPEG_PATH") +else: + _ffmpeg_paths = [] + try: + from imageio_ffmpeg import get_ffmpeg_exe + _ffmpeg_paths.append(get_ffmpeg_exe()) + except: + pass + system_ffmpeg = _shutil.which("ffmpeg") + if system_ffmpeg: + _ffmpeg_paths.append(system_ffmpeg) + if os.path.isfile("ffmpeg"): _ffmpeg_paths.append(os.path.abspath("ffmpeg")) + if os.path.isfile("ffmpeg.exe"): _ffmpeg_paths.append(os.path.abspath("ffmpeg.exe")) + _ffmpeg = _ffmpeg_paths[0] if _ffmpeg_paths else None + +import json as _json +import datetime as _dt +import sys as _sys +import functools as _functools +from PIL import Image as _PILImage, ExifTags as _ExifTags +from PIL.PngImagePlugin import PngInfo as _PngInfo +from string import Template as _Template + +# gifski_path +_gifski_path = os.environ.get("VHS_GIFSKI", None) +if _gifski_path is None: + _gifski_path = os.environ.get("JOV_GIFSKI", None) + if _gifski_path is None: + _gifski_path = _shutil.which("gifski") + +def _merge_filter_args(args, ftype="-vf"): + try: + start_index = args.index(ftype)+1 + index = start_index + while True: + index = args.index(ftype, index) + args[start_index] += ',' + args[index+1] + args.pop(index) + args.pop(index) + except ValueError: + pass + +def _cached(duration): + def dec(f): + cached_ret = None + cache_time = 0 + def cached_func(): + nonlocal cache_time, cached_ret + if time.time() > cache_time + duration or cached_ret is None: + cache_time = time.time() + cached_ret = f() + return cached_ret + return cached_func + return dec + +class _MultiInput(str): + def __new__(cls, string, allowed_types="*"): + res = super().__new__(cls, string) + res.allowed_types=allowed_types + return res + def __ne__(self, other): + if self.allowed_types == "*" or other == "*": + return False + return other not in self.allowed_types + +_imageOrLatent = _MultiInput("IMAGE", ["IMAGE", "LATENT"]) +_floatOrInt = _MultiInput("FLOAT", ["FLOAT", "INT"]) + +class _ContainsAll(dict): + def __contains__(self, other): + return True + def __getitem__(self, key): + return super().get(key, (None, {})) + +def _flatten_list(l): + ret = [] + for e in l: + if isinstance(e, list): + ret.extend(e) + else: + ret.append(e) + return ret + +def _iterate_format(video_format, for_widgets=True): + def indirector(cont, index): + if isinstance(cont[index], list) and (not for_widgets + or len(cont[index])> 1 and not isinstance(cont[index][1], dict)): + inp = yield cont[index] + if inp is not None: + cont[index] = inp + yield + for k in video_format: + if k == "extra_widgets": + if for_widgets: + yield from video_format["extra_widgets"] + elif k.endswith("_pass"): + for i in range(len(video_format[k])): + yield from indirector(video_format[k], i) + if not for_widgets: + video_format[k] = _flatten_list(video_format[k]) + else: + yield from indirector(video_format, k) + +if 'VHS_video_formats' not in folder_paths.folder_names_and_paths: + folder_paths.folder_names_and_paths["VHS_video_formats"] = ((),{".json"}) +if len(folder_paths.folder_names_and_paths['VHS_video_formats'][1]) == 0: + folder_paths.folder_names_and_paths["VHS_video_formats"][1].add(".json") + +_base_formats_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), "video_formats") + +@_cached(5) +def _get_video_formats(): + format_files = {} + for format_name in folder_paths.get_filename_list("VHS_video_formats"): + format_files[format_name] = folder_paths.get_full_path("VHS_video_formats", format_name) + os.makedirs(_base_formats_dir, exist_ok=True) + for item in os.scandir(_base_formats_dir): + if not item.is_file() or not item.name.endswith('.json'): + continue + format_files[item.name[:-5]] = item.path + formats = [] + format_widgets = {} + for format_name, path in format_files.items(): + with open(path, 'r') as stream: + video_format = _json.load(stream) + if "gifski_pass" in video_format and _gifski_path is None: + continue + widgets = list(_iterate_format(video_format)) + formats.append("video/" + format_name) + if (len(widgets) > 0): + format_widgets["video/"+ format_name] = widgets + return formats, format_widgets + +def _apply_format_widgets(format_name, kwargs): + if os.path.exists(os.path.join(_base_formats_dir, format_name + ".json")): + video_format_path = os.path.join(_base_formats_dir, format_name + ".json") + else: + video_format_path = folder_paths.get_full_path("VHS_video_formats", format_name) + with open(video_format_path, 'r') as stream: + video_format = _json.load(stream) + for w in _iterate_format(video_format): + if w[0] not in kwargs: + if len(w) > 2 and 'default' in w[2]: + default = w[2]['default'] + else: + if type(w[1]) is list: + default = w[1][0] + else: + default = {"BOOLEAN": False, "INT": 0, "FLOAT": 0, "STRING": ""}[w[1]] + kwargs[w[0]] = default + wit = _iterate_format(video_format, False) + for w in wit: + while isinstance(w, list): + if len(w) == 1: + w = [_Template(x).substitute(**kwargs) for x in w[0]] + break + elif isinstance(w[1], dict): + w = w[1][str(kwargs[w[0]])] + elif len(w) > 3: + w = _Template(w[3]).substitute(val=kwargs[w[0]]) + else: + w = str(kwargs[w[0]]) + wit.send(w) + return video_format + +def _tensor_to_int(tensor, bits): + tensor = tensor.cpu().numpy() * (2**bits-1) + 0.5 + return np.clip(tensor, 0, (2**bits-1)) +def _tensor_to_shorts(tensor): + return _tensor_to_int(tensor, 16).astype(np.uint16) +def _tensor_to_bytes(tensor): + return _tensor_to_int(tensor, 8).astype(np.uint8) + +def _ffmpeg_process(args, video_format, video_metadata, file_path, env): + res = None + frame_data = yield + total_frames_output = 0 + if video_format.get('save_metadata', 'False') != 'False': + os.makedirs(folder_paths.get_temp_directory(), exist_ok=True) + metadata_path = os.path.join(folder_paths.get_temp_directory(), "metadata.txt") + def escape_ffmpeg_metadata(key, value): + value = str(value) + value = value.replace("\\","\\\\") + value = value.replace(";","\\;") + value = value.replace("#","\\#") + value = value.replace("=","\\=") + value = value.replace("\n","\\\n") + return f"{key}={value}" + with open(metadata_path, "w") as f: + f.write(";FFMETADATA1\n") + if "prompt" in video_metadata: + f.write(escape_ffmpeg_metadata("prompt", _json.dumps(video_metadata["prompt"])) + "\n") + if "workflow" in video_metadata: + f.write(escape_ffmpeg_metadata("workflow", _json.dumps(video_metadata["workflow"])) + "\n") + for k, v in video_metadata.items(): + if k not in ["prompt", "workflow"]: + f.write(escape_ffmpeg_metadata(k, _json.dumps(v)) + "\n") + m_args = args[:1] + ["-i", metadata_path] + args[1:] + ["-metadata", "creation_time=now", "-movflags", "use_metadata_tags"] + with subprocess.Popen(m_args + [file_path], stderr=subprocess.PIPE, + stdin=subprocess.PIPE, env=env) as proc: + try: + while frame_data is not None: + proc.stdin.write(frame_data) + frame_data = yield + total_frames_output+=1 + proc.stdin.flush() + proc.stdin.close() + res = proc.stderr.read() + except BrokenPipeError as e: + err = proc.stderr.read() + if os.path.exists(file_path): + raise Exception("An error occurred in the ffmpeg subprocess:\n" + err.decode(*ENCODE_ARGS)) + print(err.decode(*ENCODE_ARGS), end="", file=_sys.stderr) + if res != b'': + with subprocess.Popen(args + [file_path], stderr=subprocess.PIPE, + stdin=subprocess.PIPE, env=env) as proc: + try: + while frame_data is not None: + proc.stdin.write(frame_data) + frame_data = yield + total_frames_output+=1 + proc.stdin.flush() + proc.stdin.close() + res = proc.stderr.read() + except BrokenPipeError as e: + res = proc.stderr.read() + raise Exception("An error occurred in the ffmpeg subprocess:\n" + res.decode(*ENCODE_ARGS)) + yield total_frames_output + if len(res) > 0: + print(res.decode(*ENCODE_ARGS), end="", file=_sys.stderr) + +def _gifski_process(args, dimensions, frame_rate, video_format, file_path, env): + frame_data = yield + with subprocess.Popen(args + video_format['main_pass'] + ['-f', 'yuv4mpegpipe', '-'], + stderr=subprocess.PIPE, stdin=subprocess.PIPE, + stdout=subprocess.PIPE, env=env) as procff: + with subprocess.Popen([_gifski_path] + video_format['gifski_pass'] + + ['-W', f'{dimensions[0]}', '-H', f'{dimensions[1]}'] + + ['-r', f'{frame_rate}'] + + ['-q', '-o', file_path, '-'], stderr=subprocess.PIPE, + stdin=procff.stdout, stdout=subprocess.PIPE, + env=env) as procgs: + try: + while frame_data is not None: + procff.stdin.write(frame_data) + frame_data = yield + procff.stdin.flush() + procff.stdin.close() + resff = procff.stderr.read() + resgs = procgs.stderr.read() + outgs = procgs.stdout.read() + except BrokenPipeError as e: + procff.stdin.close() + resff = procff.stderr.read() + resgs = procgs.stderr.read() + raise Exception("An error occurred while creating gifski output\n" + + "Make sure you are using gifski --version >=1.32.0\nffmpeg: " + + resff.decode(*ENCODE_ARGS) + '\ngifski: ' + resgs.decode(*ENCODE_ARGS)) + if len(resff) > 0: + print(resff.decode(*ENCODE_ARGS), end="", file=_sys.stderr) + if len(resgs) > 0: + print(resgs.decode(*ENCODE_ARGS), end="", file=_sys.stderr) + if len(outgs) > 0: + print(outgs.decode(*ENCODE_ARGS)) + +def _to_pingpong(inp): + if not hasattr(inp, "__getitem__"): + inp = list(inp) + yield from inp + for i in range(len(inp)-2,0,-1): + yield inp[i] + + +# ============================================================ +# XB_VideoCombine — 完整搬运 VHS VideoCombine，完全独立 +# ============================================================ + +class XB_VideoCombine: + @classmethod + def INPUT_TYPES(cls): + ffmpeg_formats, format_widgets = _get_video_formats() + format_widgets["image/webp"] = [['lossless', "BOOLEAN", {'default': True}]] + return { + "required": { + "images": (_imageOrLatent,), + "frame_rate": (_floatOrInt, {"default": 8, "min": 1, "step": 1}), + "loop_count": ("INT", {"default": 0, "min": 0, "max": 100, "step": 1}), + "filename_prefix": ("STRING", {"default": "视频"}), + "format": (["image/gif", "image/webp"] + ffmpeg_formats, {'formats': format_widgets}), + "pingpong": ("BOOLEAN", {"default": False}), + "save_output": ("BOOLEAN", {"default": True}), + }, + "optional": { + "audio": ("AUDIO",), + }, + "hidden": _ContainsAll({ + "prompt": "PROMPT", + "extra_pnginfo": "EXTRA_PNGINFO", + "unique_id": "UNIQUE_ID" + }), + } + + RETURN_TYPES = ("VHS_FILENAMES",) + RETURN_NAMES = ("Filenames",) + OUTPUT_NODE = True + FUNCTION = "combine_video" + CATEGORY = "XB_ToolBox/Video" + + def combine_video(self, frame_rate, loop_count, images=None, latents=None, + filename_prefix="视频", format="image/gif", pingpong=False, + save_output=True, prompt=None, extra_pnginfo=None, + audio=None, unique_id=None, manual_format_widgets=None, **kwargs): + if latents is not None: + images = latents + if images is None: + return ((save_output, []),) + if isinstance(images, torch.Tensor) and images.size(0) == 0: + return ((save_output, []),) + num_frames = len(images) + pbar = ProgressBar(num_frames) + first_image = images[0] + while len(first_image.shape) > 3: + first_image = first_image[0] + images = iter(images) + + output_dir = folder_paths.get_output_directory() if save_output else folder_paths.get_temp_directory() + # 展开 %date:yyyy-MM-dd% 之类的标记，否则会生成非法目录名（Windows: WinError 267） + filename_prefix = resolve_filename_prefix(filename_prefix) + full_output_folder, filename, _, subfolder, _ = folder_paths.get_save_image_path(filename_prefix, output_dir) + output_files = [] + + metadata = _PngInfo() + video_metadata = {} + if prompt is not None: + metadata.add_text("prompt", _json.dumps(prompt)) + video_metadata["prompt"] = prompt + if extra_pnginfo is not None: + for x in extra_pnginfo: + metadata.add_text(x, _json.dumps(extra_pnginfo[x])) + video_metadata[x] = extra_pnginfo[x] + extra_options = extra_pnginfo.get('workflow', {}).get('extra', {}) + else: + extra_options = {} + metadata.add_text("CreationTime", _dt.datetime.now().isoformat(" ")[:19]) + + max_counter = 0 + matcher = re.compile(f"{re.escape(filename)}_(\\d+)\\D*\\..+", re.IGNORECASE) + for existing_file in os.listdir(full_output_folder): + match = matcher.fullmatch(existing_file) + if match: + max_counter = max(max_counter, int(match.group(1))) + counter = max_counter + 1 + + first_image_file = f"{filename}_{counter:05}.png" + file_path = os.path.join(full_output_folder, first_image_file) + if extra_options.get('VHS_MetadataImage', True) != False: + _PILImage.fromarray(_tensor_to_bytes(first_image)).save(file_path, pnginfo=metadata, compress_level=4) + output_files.append(file_path) + + format_type, format_ext = format.split("/") + if format_type == "image": + image_kwargs = {} + if format_ext == "gif": + image_kwargs['disposal'] = 2 + if format_ext == "webp": + exif = _PILImage.Exif() + exif[_ExifTags.IFD.Exif] = {36867: _dt.datetime.now().isoformat(" ")[:19]} + image_kwargs['exif'] = exif + image_kwargs['lossless'] = kwargs.get("lossless", True) + file = f"{filename}_{counter:05}.{format_ext}" + file_path = os.path.join(full_output_folder, file) + if pingpong: + images = _to_pingpong(images) + def frames_gen(images): + for i in images: + pbar.update(1) + yield _PILImage.fromarray(_tensor_to_bytes(i)) + frames = frames_gen(images) + next(frames).save(file_path, format=format_ext.upper(), save_all=True, + append_images=frames, duration=round(1000/frame_rate), + loop=loop_count, compress_level=4, **image_kwargs) + output_files.append(file_path) + else: + if _ffmpeg is None: + raise RuntimeError("ffmpeg 未找到") + + if manual_format_widgets is not None: + kwargs.update(manual_format_widgets) + + has_alpha = first_image.shape[-1] == 4 + kwargs["has_alpha"] = has_alpha + video_format = _apply_format_widgets(format_ext, kwargs) + dim_alignment = video_format.get("dim_alignment", 2) + if (first_image.shape[1] % dim_alignment) or (first_image.shape[0] % dim_alignment): + to_pad = (-first_image.shape[1] % dim_alignment, -first_image.shape[0] % dim_alignment) + padding = (to_pad[0]//2, to_pad[0] - to_pad[0]//2, to_pad[1]//2, to_pad[1] - to_pad[1]//2) + padfunc = torch.nn.ReplicationPad2d(padding) + def pad(image): + image = image.permute((2,0,1)) + padded = padfunc(image.to(dtype=torch.float32)) + return padded.permute((1,2,0)) + images = map(pad, images) + dimensions = (-first_image.shape[1] % dim_alignment + first_image.shape[1], + -first_image.shape[0] % dim_alignment + first_image.shape[0]) + else: + dimensions = (first_image.shape[1], first_image.shape[0]) + if pingpong: + images = _to_pingpong(images) + if num_frames > 2: + num_frames += num_frames -2 + pbar.total = num_frames + if loop_count > 0: + loop_args = ["-vf", "loop=loop=" + str(loop_count)+":size=" + str(num_frames)] + else: + loop_args = [] + if video_format.get('input_color_depth', '8bit') == '16bit': + images = map(_tensor_to_shorts, images) + i_pix_fmt = 'rgba64' if has_alpha else 'rgb48' + else: + images = map(_tensor_to_bytes, images) + i_pix_fmt = 'rgba' if has_alpha else 'rgb24' + file = f"{filename}_{counter:05}.{video_format['extension']}" + file_path = os.path.join(full_output_folder, file) + bitrate_arg = [] + bitrate = video_format.get('bitrate') + if bitrate is not None: + bitrate_arg = ["-b:v", str(bitrate) + "M" if video_format.get('megabit') == 'True' else str(bitrate) + "K"] + args = [_ffmpeg, "-v", "error", "-f", "rawvideo", "-pix_fmt", i_pix_fmt, + "-color_range", "pc", "-colorspace", "rgb", "-color_primaries", "bt709", + "-color_trc", video_format.get("fake_trc", "iec61966-2-1"), + "-s", f"{dimensions[0]}x{dimensions[1]}", "-r", str(frame_rate), "-i", "-"] + loop_args + + images = map(lambda x: x.tobytes(), images) + env = os.environ.copy() + if "environment" in video_format: + env.update(video_format["environment"]) + + if "pre_pass" in video_format: + images = [b''.join(images)] + os.makedirs(folder_paths.get_temp_directory(), exist_ok=True) + in_args_len = args.index("-i") + 2 + pre_pass_args = args[:in_args_len] + video_format['pre_pass'] + _merge_filter_args(pre_pass_args) + try: + subprocess.run(pre_pass_args, input=images[0], env=env, capture_output=True, check=True) + except subprocess.CalledProcessError as e: + raise Exception("An error occurred in the ffmpeg prepass:\n" + e.stderr.decode(*ENCODE_ARGS)) + if "inputs_main_pass" in video_format: + in_args_len = args.index("-i") + 2 + args = args[:in_args_len] + video_format['inputs_main_pass'] + args[in_args_len:] + + if 'gifski_pass' in video_format: + format = 'image/gif' + output_process = _gifski_process(args, dimensions, frame_rate, video_format, file_path, env) + audio = None + else: + args += video_format['main_pass'] + bitrate_arg + _merge_filter_args(args) + output_process = _ffmpeg_process(args, video_format, video_metadata, file_path, env) + output_process.send(None) + + for image in images: + pbar.update(1) + output_process.send(image) + try: + total_frames_output = output_process.send(None) + output_process.send(None) + except StopIteration: + pass + output_files.append(file_path) + + a_waveform = None + if audio is not None: + try: + a_waveform = audio['waveform'] + except: + pass + if a_waveform is not None: + output_file_with_audio = f"{filename}_{counter:05}-audio.{video_format['extension']}" + output_file_with_audio_path = os.path.join(full_output_folder, output_file_with_audio) + if "audio_pass" not in video_format: + video_format["audio_pass"] = ["-c:a", "libopus"] + channels = audio['waveform'].size(1) + min_audio_dur = total_frames_output / frame_rate + 1 + if video_format.get('trim_to_audio', 'False') != 'False': + apad = [] + else: + apad = ["-af", "apad=whole_dur="+str(min_audio_dur)] + mux_args = [_ffmpeg, "-v", "error", "-n", "-i", file_path, + "-ar", str(audio['sample_rate']), "-ac", str(channels), + "-f", "f32le", "-i", "-", "-c:v", "copy"] \ + + video_format["audio_pass"] + apad + ["-shortest", output_file_with_audio_path] + audio_data = audio['waveform'].squeeze(0).transpose(0,1).numpy().tobytes() + _merge_filter_args(mux_args, '-af') + try: + res = subprocess.run(mux_args, input=audio_data, env=env, capture_output=True, check=True) + except subprocess.CalledProcessError as e: + raise Exception("An error occured in the ffmpeg subprocess:\n" + e.stderr.decode(*ENCODE_ARGS)) + output_files.append(output_file_with_audio_path) + file = output_file_with_audio + if extra_options.get('VHS_KeepIntermediate', True) == False: + for intermediate in output_files[1:-1]: + if os.path.exists(intermediate): + os.remove(intermediate) + preview = { + "filename": file, + "subfolder": subfolder, + "type": "output" if save_output else "temp", + "format": format, + "frame_rate": frame_rate, + "workflow": first_image_file, + "fullpath": output_files[-1], + } + if num_frames == 1 and 'png' in format and '%03d' in file: + preview['format'] = 'image/png' + preview['filename'] = file.replace('%03d', '001') return {"ui": {"gifs": [preview]}, "result": ((save_output, output_files),)} \ No newline at end of file diff --git a/nodes_wan.py b/nodes/video/nodes_wan.py similarity index 98% rename from nodes_wan.py rename to nodes/video/nodes_wan.py index d1c40f2..b5cb44b 100644 --- a/nodes_wan.py +++ b/nodes/video/nodes_wan.py @@ -1,218 +1,218 @@ -""" -XB-ToolBox Wan 视频节点 (ROCm 优化版) -===================================== -完全独立：XB_WanAnimateToVideo / CompileSettings / BlockSwap 零依赖。 -ModelLoader / Sampler / VAELoader / Decode / TextEncode 需要 ComfyUI-WanVideoWrapper。 -""" -import torch -import os -import sys -import gc -import importlib.util -import comfy.model_management as mm -import folder_paths - -_SCALE_METHODS = ["lanczos", "bilinear", "bicubic", "nearest-exact", "area"] -_CROP_MODES = ["disabled", "center"] - -# ROCm 全局补丁（纯 torch，零依赖） -if not hasattr(torch.backends.cuda, "matmul"): - class _FM: allow_fp16_accumulation = False - torch.backends.cuda.matmul = _FM() -elif not hasattr(torch.backends.cuda.matmul, "allow_fp16_accumulation"): - torch.backends.cuda.matmul.allow_fp16_accumulation = False -if not hasattr(torch.cuda, "get_device_capability"): - torch.cuda.get_device_capability = lambda d=None: (9, 0) -if not hasattr(torch.cuda, "reset_peak_memory_stats"): - torch.cuda.reset_peak_memory_stats = lambda d=None: None - -_PKG_NAME = "_xb_wanwrapper" -_wan_loaded = False - - -def _find_wan_root(): - for base in folder_paths.get_folder_paths("custom_nodes"): - for n in os.listdir(base): - if n.startswith("ComfyUI-WanVideoWrapper"): - p = os.path.join(base, n) - if os.path.exists(os.path.join(p, "__init__.py")): - return p - raise RuntimeError( - "ComfyUI-WanVideoWrapper 未安装。\n" - "此节点依赖它作为模型架构库。\n" - "XB_WanAnimateToVideo/XB_WanCompileSettings 不需要。" - ) - - -def _wan_setup(): - global _wan_loaded - if _wan_loaded: return - root = _find_wan_root() - if _PKG_NAME not in sys.modules: - s = importlib.util.spec_from_file_location(_PKG_NAME, os.path.join(root, "__init__.py"), submodule_search_locations=[root]) - m = importlib.util.module_from_spec(s) - sys.modules[_PKG_NAME] = m - s.loader.exec_module(m) - _wan_loaded = True - - -def _wan_import(rel_path: str): - _wan_setup() - full = _PKG_NAME + "." + rel_path - if full in sys.modules: return sys.modules[full] - root = _find_wan_root() - parts = rel_path.split(".") - fp = os.path.join(root, *parts[:-1], parts[-1] + ".py") - if not os.path.exists(fp): - fp = os.path.join(root, *parts, "__init__.py") - s = importlib.util.spec_from_file_location(full, fp) - m = importlib.util.module_from_spec(s) - sys.modules[full] = m - s.loader.exec_module(m) - return m - - -# ============================================================ -# XB_WanCompileSettings — Wan 编译设置 -# ============================================================ -class XB_WanCompileSettings: - @classmethod - def INPUT_TYPES(s): - return {"required": { - "backend": (["inductor"], {"default": "inductor"}), - "fullgraph": ("BOOLEAN", {"default": False}), - "mode": (["default", "reduce-overhead", "max-autotune"], {"default": "default"}), - "dynamic": ("BOOLEAN", {"default": False}), - "dynamo_cache_size_limit": ("INT", {"default": 64, "min": 0, "max": 1024}), - "compile_transformer_blocks_only": ("BOOLEAN", {"default": True}), - }} - RETURN_TYPES = ("WANCOMPILEARGS",) - FUNCTION = "go" - CATEGORY = "XB_ToolBox/Wan" - def go(self, **kw): - kw.setdefault("force_parameter_static_shapes", False) - kw.setdefault("allow_unmerged_lora_compile", False) - return (kw,) - - -# ============================================================ -# XB_WanAnimateToVideo — Wan 动画转视频 -# ============================================================ -class XB_WanAnimateToVideo: - @classmethod - def INPUT_TYPES(s): - return { - "required": { - "positive": ("CONDITIONING",), "negative": ("CONDITIONING",), "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 16384, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 16384, "step": 16}), - "length": ("INT", {"default": 77, "min": 1, "max": 16384, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "continue_motion_max_frames": ("INT", {"default": 5, "min": 1, "max": 16384, "step": 4}), - "video_frame_offset": ("INT", {"default": 0, "min": 0, "max": 16384, "step": 1}), - "vae_tile_size": ("INT", {"default": 256, "min": 0, "max": 2048, "step": 64}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "reference_image": ("IMAGE",), "face_video": ("IMAGE",), - "pose_video": ("IMAGE",), "background_video": ("IMAGE",), - "character_mask": ("MASK",), "continue_motion": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - }, - } - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT", "INT", "INT") - RETURN_NAMES = ("positive", "negative", "latent", "trim_latent", "trim_image", "video_frame_offset") - FUNCTION = "go" - CATEGORY = "XB_ToolBox/Wan" - - def go(self, positive, negative, vae, width, height, length, batch_size, - continue_motion_max_frames, video_frame_offset, vae_tile_size=256, - reference_image=None, clip_vision_output=None, face_video=None, - pose_video=None, continue_motion=None, background_video=None, character_mask=None, - scale_method="lanczos", crop_mode="center"): - import comfy.utils - import node_helpers - def _enc(pixels): - if pixels.dim() == 3: pixels = pixels.unsqueeze(0) - p = pixels[:, :, :, :3] - - # 🛡️ 显存连续性重组 - if not p.is_contiguous(): p = p.contiguous() - # 🛡️ VAE 精度对齐 - if hasattr(vae, 'first_stage_model'): - vae_dtype = getattr(vae.first_stage_model, 'dtype', None) - if vae_dtype is None: - try: vae_dtype = next(vae.first_stage_model.parameters()).dtype - except: pass - if vae_dtype is not None and p.dtype != vae_dtype: - p = p.to(vae_dtype) - - if vae_tile_size and vae_tile_size > 0: - try: return vae.encode_tiled(p, tile_x=vae_tile_size, tile_y=vae_tile_size, overlap=32, tile_t=32, overlap_t=4) - except (AttributeError, TypeError): pass - return vae.encode(p) - trim_latent, ref_motion_latent_length = 0, 0 - latent_length = ((length - 1) // 4) + 1 - if reference_image is None: reference_image = torch.zeros((1, height, width, 3)) - image = comfy.utils.common_upscale(reference_image[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - concat_latent_image = _enc(image) - mask = torch.zeros((1, 4, concat_latent_image.shape[-3], concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=concat_latent_image.device, dtype=concat_latent_image.dtype) - trim_latent += concat_latent_image.shape[2] - if continue_motion is None: - image = torch.ones((length, height, width, 3)) * 0.5 - else: - continue_motion = continue_motion[-continue_motion_max_frames:] - video_frame_offset = max(0, video_frame_offset - continue_motion.shape[0]) - continue_motion = comfy.utils.common_upscale(continue_motion[-length:].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - image = torch.ones((length, height, width, continue_motion.shape[-1]), device=continue_motion.device, dtype=continue_motion.dtype) * 0.5 - image[:continue_motion.shape[0]] = continue_motion - ref_motion_latent_length += ((continue_motion.shape[0] - 1) // 4) + 1 - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - if pose_video is not None: - if pose_video.shape[0] <= video_frame_offset: pose_video = None - else: pose_video = pose_video[video_frame_offset:] - if pose_video is not None: - pose_video = comfy.utils.common_upscale(pose_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - if pose_video.shape[0] < length: pose_video = torch.cat((pose_video,) + (pose_video[-1:],) * (length - pose_video.shape[0]), dim=0) - pvl = _enc(pose_video) - positive = node_helpers.conditioning_set_values(positive, {"pose_video_latent": pvl}) - negative = node_helpers.conditioning_set_values(negative, {"pose_video_latent": pvl}) - if face_video is not None: - if face_video.shape[0] <= video_frame_offset: face_video = None - else: face_video = face_video[video_frame_offset:] - if face_video is not None: - fv = comfy.utils.common_upscale(face_video[:length].movedim(-1, 1), 512, 512, "area", "center") * 2.0 - 1.0 - fv = fv.movedim(0, 1).unsqueeze(0) - positive = node_helpers.conditioning_set_values(positive, {"face_video_pixels": fv}) - negative = node_helpers.conditioning_set_values(negative, {"face_video_pixels": fv * 0.0 - 1.0}) - ref_images_num = max(0, ref_motion_latent_length * 4 - 3) - if background_video is not None and background_video.shape[0] > video_frame_offset: - bg = comfy.utils.common_upscale(background_video[video_frame_offset:][:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - if bg.shape[0] > ref_images_num: image[ref_images_num:bg.shape[0]] = bg[ref_images_num:] - mask_ref = torch.ones((1, 1, latent_length * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=mask.device, dtype=mask.dtype) - if continue_motion is not None: mask_ref[:, :, :ref_motion_latent_length * 4] = 0.0 - if character_mask is not None: - if character_mask.shape[0] > video_frame_offset or character_mask.shape[0] == 1: - cm = character_mask.repeat((length,) + (1,) * (character_mask.ndim - 1)) if character_mask.shape[0] == 1 else character_mask[video_frame_offset:] - # cm: [T, H, W] (3D) → 保持4D传入 common_upscale - if cm.ndim == 3: - cm = cm.unsqueeze(1) # [T, 1, H, W] - cm = comfy.utils.common_upscale(cm[:length], concat_latent_image.shape[-1], concat_latent_image.shape[-2], "nearest-exact", "center") - # cm: [length, 1, H_lat, W_lat] → 转为 [1, 1, length, H_lat, W_lat] - cm = cm.permute(1, 0, 2, 3).unsqueeze(0) - if cm.shape[2] > ref_images_num: mask_ref[:, :, ref_images_num:cm.shape[2]] = cm[:, :, ref_images_num:] - concat_latent_image = torch.cat((concat_latent_image, _enc(image)), dim=2) - mask_ref = mask_ref.view(1, mask_ref.shape[2] // 4, 4, mask_ref.shape[3], mask_ref.shape[4]).transpose(1, 2) - mask = torch.cat((mask, mask_ref), dim=2) - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - latent = torch.zeros([batch_size, 16, latent_length + trim_latent, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - return (positive, negative, {"samples": latent}, trim_latent, max(0, ref_motion_latent_length * 4 - 3), video_frame_offset + length) - - -__all__ = [ - "XB_WanCompileSettings", "XB_WanAnimateToVideo", -] +""" +XB-ToolBox Wan 视频节点 (ROCm 优化版) +===================================== +完全独立：XB_WanAnimateToVideo / CompileSettings / BlockSwap 零依赖。 +ModelLoader / Sampler / VAELoader / Decode / TextEncode 需要 ComfyUI-WanVideoWrapper。 +""" +import torch +import os +import sys +import gc +import importlib.util +import comfy.model_management as mm +import folder_paths + +_SCALE_METHODS = ["lanczos", "bilinear", "bicubic", "nearest-exact", "area"] +_CROP_MODES = ["disabled", "center"] + +# ROCm 全局补丁（纯 torch，零依赖） +if not hasattr(torch.backends.cuda, "matmul"): + class _FM: allow_fp16_accumulation = False + torch.backends.cuda.matmul = _FM() +elif not hasattr(torch.backends.cuda.matmul, "allow_fp16_accumulation"): + torch.backends.cuda.matmul.allow_fp16_accumulation = False +if not hasattr(torch.cuda, "get_device_capability"): + torch.cuda.get_device_capability = lambda d=None: (9, 0) +if not hasattr(torch.cuda, "reset_peak_memory_stats"): + torch.cuda.reset_peak_memory_stats = lambda d=None: None + +_PKG_NAME = "_xb_wanwrapper" +_wan_loaded = False + + +def _find_wan_root(): + for base in folder_paths.get_folder_paths("custom_nodes"): + for n in os.listdir(base): + if n.startswith("ComfyUI-WanVideoWrapper"): + p = os.path.join(base, n) + if os.path.exists(os.path.join(p, "__init__.py")): + return p + raise RuntimeError( + "ComfyUI-WanVideoWrapper 未安装。\n" + "此节点依赖它作为模型架构库。\n" + "XB_WanAnimateToVideo/XB_WanCompileSettings 不需要。" + ) + + +def _wan_setup(): + global _wan_loaded + if _wan_loaded: return + root = _find_wan_root() + if _PKG_NAME not in sys.modules: + s = importlib.util.spec_from_file_location(_PKG_NAME, os.path.join(root, "__init__.py"), submodule_search_locations=[root]) + m = importlib.util.module_from_spec(s) + sys.modules[_PKG_NAME] = m + s.loader.exec_module(m) + _wan_loaded = True + + +def _wan_import(rel_path: str): + _wan_setup() + full = _PKG_NAME + "." + rel_path + if full in sys.modules: return sys.modules[full] + root = _find_wan_root() + parts = rel_path.split(".") + fp = os.path.join(root, *parts[:-1], parts[-1] + ".py") + if not os.path.exists(fp): + fp = os.path.join(root, *parts, "__init__.py") + s = importlib.util.spec_from_file_location(full, fp) + m = importlib.util.module_from_spec(s) + sys.modules[full] = m + s.loader.exec_module(m) + return m + + +# ============================================================ +# XB_WanCompileSettings — Wan 编译设置 +# ============================================================ +class XB_WanCompileSettings: + @classmethod + def INPUT_TYPES(s): + return {"required": { + "backend": (["inductor"], {"default": "inductor"}), + "fullgraph": ("BOOLEAN", {"default": False}), + "mode": (["default", "reduce-overhead", "max-autotune"], {"default": "default"}), + "dynamic": ("BOOLEAN", {"default": False}), + "dynamo_cache_size_limit": ("INT", {"default": 64, "min": 0, "max": 1024}), + "compile_transformer_blocks_only": ("BOOLEAN", {"default": True}), + }} + RETURN_TYPES = ("WANCOMPILEARGS",) + FUNCTION = "go" + CATEGORY = "XB_ToolBox/Wan" + def go(self, **kw): + kw.setdefault("force_parameter_static_shapes", False) + kw.setdefault("allow_unmerged_lora_compile", False) + return (kw,) + + +# ============================================================ +# XB_WanAnimateToVideo — Wan 动画转视频 +# ============================================================ +class XB_WanAnimateToVideo: + @classmethod + def INPUT_TYPES(s): + return { + "required": { + "positive": ("CONDITIONING",), "negative": ("CONDITIONING",), "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 16384, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 16384, "step": 16}), + "length": ("INT", {"default": 77, "min": 1, "max": 16384, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "continue_motion_max_frames": ("INT", {"default": 5, "min": 1, "max": 16384, "step": 4}), + "video_frame_offset": ("INT", {"default": 0, "min": 0, "max": 16384, "step": 1}), + "vae_tile_size": ("INT", {"default": 256, "min": 0, "max": 2048, "step": 64}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "reference_image": ("IMAGE",), "face_video": ("IMAGE",), + "pose_video": ("IMAGE",), "background_video": ("IMAGE",), + "character_mask": ("MASK",), "continue_motion": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + }, + } + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT", "INT", "INT") + RETURN_NAMES = ("positive", "negative", "latent", "trim_latent", "trim_image", "video_frame_offset") + FUNCTION = "go" + CATEGORY = "XB_ToolBox/Wan" + + def go(self, positive, negative, vae, width, height, length, batch_size, + continue_motion_max_frames, video_frame_offset, vae_tile_size=256, + reference_image=None, clip_vision_output=None, face_video=None, + pose_video=None, continue_motion=None, background_video=None, character_mask=None, + scale_method="lanczos", crop_mode="center"): + import comfy.utils + import node_helpers + def _enc(pixels): + if pixels.dim() == 3: pixels = pixels.unsqueeze(0) + p = pixels[:, :, :, :3] + + # 🛡️ 显存连续性重组 + if not p.is_contiguous(): p = p.contiguous() + # 🛡️ VAE 精度对齐 + if hasattr(vae, 'first_stage_model'): + vae_dtype = getattr(vae.first_stage_model, 'dtype', None) + if vae_dtype is None: + try: vae_dtype = next(vae.first_stage_model.parameters()).dtype + except: pass + if vae_dtype is not None and p.dtype != vae_dtype: + p = p.to(vae_dtype) + + if vae_tile_size and vae_tile_size > 0: + try: return vae.encode_tiled(p, tile_x=vae_tile_size, tile_y=vae_tile_size, overlap=32, tile_t=32, overlap_t=4) + except (AttributeError, TypeError): pass + return vae.encode(p) + trim_latent, ref_motion_latent_length = 0, 0 + latent_length = ((length - 1) // 4) + 1 + if reference_image is None: reference_image = torch.zeros((1, height, width, 3)) + image = comfy.utils.common_upscale(reference_image[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + concat_latent_image = _enc(image) + mask = torch.zeros((1, 4, concat_latent_image.shape[-3], concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=concat_latent_image.device, dtype=concat_latent_image.dtype) + trim_latent += concat_latent_image.shape[2] + if continue_motion is None: + image = torch.ones((length, height, width, 3)) * 0.5 + else: + continue_motion = continue_motion[-continue_motion_max_frames:] + video_frame_offset = max(0, video_frame_offset - continue_motion.shape[0]) + continue_motion = comfy.utils.common_upscale(continue_motion[-length:].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + image = torch.ones((length, height, width, continue_motion.shape[-1]), device=continue_motion.device, dtype=continue_motion.dtype) * 0.5 + image[:continue_motion.shape[0]] = continue_motion + ref_motion_latent_length += ((continue_motion.shape[0] - 1) // 4) + 1 + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + if pose_video is not None: + if pose_video.shape[0] <= video_frame_offset: pose_video = None + else: pose_video = pose_video[video_frame_offset:] + if pose_video is not None: + pose_video = comfy.utils.common_upscale(pose_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + if pose_video.shape[0] < length: pose_video = torch.cat((pose_video,) + (pose_video[-1:],) * (length - pose_video.shape[0]), dim=0) + pvl = _enc(pose_video) + positive = node_helpers.conditioning_set_values(positive, {"pose_video_latent": pvl}) + negative = node_helpers.conditioning_set_values(negative, {"pose_video_latent": pvl}) + if face_video is not None: + if face_video.shape[0] <= video_frame_offset: face_video = None + else: face_video = face_video[video_frame_offset:] + if face_video is not None: + fv = comfy.utils.common_upscale(face_video[:length].movedim(-1, 1), 512, 512, "area", "center") * 2.0 - 1.0 + fv = fv.movedim(0, 1).unsqueeze(0) + positive = node_helpers.conditioning_set_values(positive, {"face_video_pixels": fv}) + negative = node_helpers.conditioning_set_values(negative, {"face_video_pixels": fv * 0.0 - 1.0}) + ref_images_num = max(0, ref_motion_latent_length * 4 - 3) + if background_video is not None and background_video.shape[0] > video_frame_offset: + bg = comfy.utils.common_upscale(background_video[video_frame_offset:][:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + if bg.shape[0] > ref_images_num: image[ref_images_num:bg.shape[0]] = bg[ref_images_num:] + mask_ref = torch.ones((1, 1, latent_length * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=mask.device, dtype=mask.dtype) + if continue_motion is not None: mask_ref[:, :, :ref_motion_latent_length * 4] = 0.0 + if character_mask is not None: + if character_mask.shape[0] > video_frame_offset or character_mask.shape[0] == 1: + cm = character_mask.repeat((length,) + (1,) * (character_mask.ndim - 1)) if character_mask.shape[0] == 1 else character_mask[video_frame_offset:] + # cm: [T, H, W] (3D) → 保持4D传入 common_upscale + if cm.ndim == 3: + cm = cm.unsqueeze(1) # [T, 1, H, W] + cm = comfy.utils.common_upscale(cm[:length], concat_latent_image.shape[-1], concat_latent_image.shape[-2], "nearest-exact", "center") + # cm: [length, 1, H_lat, W_lat] → 转为 [1, 1, length, H_lat, W_lat] + cm = cm.permute(1, 0, 2, 3).unsqueeze(0) + if cm.shape[2] > ref_images_num: mask_ref[:, :, ref_images_num:cm.shape[2]] = cm[:, :, ref_images_num:] + concat_latent_image = torch.cat((concat_latent_image, _enc(image)), dim=2) + mask_ref = mask_ref.view(1, mask_ref.shape[2] // 4, 4, mask_ref.shape[3], mask_ref.shape[4]).transpose(1, 2) + mask = torch.cat((mask, mask_ref), dim=2) + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + latent = torch.zeros([batch_size, 16, latent_length + trim_latent, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + return (positive, negative, {"samples": latent}, trim_latent, max(0, ref_motion_latent_length * 4 - 3), video_frame_offset + length) + + +__all__ = [ + "XB_WanCompileSettings", "XB_WanAnimateToVideo", +] diff --git a/nodes_wan_vae.py b/nodes/video/nodes_wan_vae.py similarity index 98% rename from nodes_wan_vae.py rename to nodes/video/nodes_wan_vae.py index d493847..3ae217d 100644 --- a/nodes_wan_vae.py +++ b/nodes/video/nodes_wan_vae.py @@ -1,1811 +1,1811 @@ -import torch -import comfy.model_management -import comfy.utils -import comfy.patcher_extension -import node_helpers -import comfy.clip_vision -import math -import numpy as np -import nodes - -_SCALE_METHODS = ["lanczos", "bilinear", "bicubic", "nearest-exact", "area"] -_CROP_MODES = ["center", "disabled"] - -def _upscale(img, w, h, method="lanczos", crop="center"): - """Upscale helper for all nodes""" - return comfy.utils.common_upscale(img.movedim(-1, 1), w, h, method, crop).movedim(1, -1) - -def _encode_vae(vae, pixels, tile_size): - if pixels.dim() == 3: - pixels = pixels.unsqueeze(0) - p = pixels[:, :, :, :3] - - # 🛡️ 显存连续性重组 (movedim/切片会产生非连续张量，MIOpen 无法处理) - if not p.is_contiguous(): - p = p.contiguous() - - # 🛡️ VAE 精度对齐 (输入 float32 vs 权重 bfloat16 会 RuntimeError) - if hasattr(vae, 'first_stage_model'): - vae_dtype = getattr(vae.first_stage_model, 'dtype', None) - if vae_dtype is None: - try: vae_dtype = next(vae.first_stage_model.parameters()).dtype - except: pass - if vae_dtype is not None and p.dtype != vae_dtype: - p = p.to(vae_dtype) - - if tile_size == 0 or tile_size is None: - return vae.encode(p) - else: - # 🔧 overlap_t = tile_t//2: Wan 3D VAE 时间卷积需要充足上下文帧， - # overlap_t=4 (12.5%) 导致分块边界 latent 衰减 → 解码尾帧发灰 - return vae.encode_tiled(p, tile_x=tile_size, tile_y=tile_size, overlap=32, tile_t=32, overlap_t=16) - -# ============================================================ -# XB_WanImageToVideo — Wan 图生视频 -# ============================================================ -class XB_WanImageToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, clip_vision_output=None, start_image=None, scale_method="lanczos", crop_mode="center"): - spacial_scale = vae.spacial_compression_encode() - latent_channels = vae.latent_channels - latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - image = torch.ones((length, height, width, start_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) * 0.5 - image[:start_image.shape[0]] = start_image - - concat_latent_image = _encode_vae(vae, image, vae_tile_size) - - mask = torch.ones((1, 1, latent.shape[2] * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) - mask[:, :, :start_image.shape[0] + 3] = 0.0 - mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) - - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - - return (positive, negative, {"samples": latent}) - - -# ============================================================ -# XB_WanFirstLastFrameToVideo — Wan 首尾帧转视频 -# ============================================================ -class XB_WanFirstLastFrameToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_start_image": ("CLIP_VISION_OUTPUT",), - "clip_vision_end_image": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "end_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, clip_vision_start_image=None, clip_vision_end_image=None, start_image=None, end_image=None, scale_method="lanczos", crop_mode="center"): - spacial_scale = vae.spacial_compression_encode() - latent = torch.zeros([batch_size, vae.latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - if end_image is not None: - end_image = _upscale(end_image[-length:], width, height, scale_method, crop_mode) - - image = torch.ones((length, height, width, 3), device=latent.device) * 0.5 - mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) - - if start_image is not None: - image[:start_image.shape[0]] = start_image - mask[:, :, :start_image.shape[0] + 3] = 0.0 - - if end_image is not None: - image[-end_image.shape[0]:] = end_image - mask[:, :, -end_image.shape[0]:] = 0.0 - - concat_latent_image = _encode_vae(vae, image, vae_tile_size) - - mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) - - clip_vision_output = None - if clip_vision_start_image is not None: - clip_vision_output = clip_vision_start_image - - if clip_vision_end_image is not None: - if clip_vision_output is not None: - states = torch.cat([clip_vision_output.penultimate_hidden_states, clip_vision_end_image.penultimate_hidden_states], dim=-2) - clip_vision_output = comfy.clip_vision.Output() - clip_vision_output.penultimate_hidden_states = states - else: - clip_vision_output = clip_vision_end_image - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - - return (positive, negative, {"samples": latent}) - - -# ============================================================ -# XB_WanFunControlToVideo — Wan FunControl 转视频 -# ============================================================ -class XB_WanFunControlToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "control_video": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, start_image=None, clip_vision_output=None, control_video=None, scale_method="lanczos", crop_mode="center"): - spacial_scale = vae.spacial_compression_encode() - latent_channels = vae.latent_channels - latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - concat_latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) - concat_latent = concat_latent.repeat(1, 2, 1, 1, 1) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - concat_latent_image = _encode_vae(vae, start_image, vae_tile_size) - concat_latent[:,latent_channels:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] - - if control_video is not None: - control_video = _upscale(control_video[:length], width, height, scale_method, crop_mode) - concat_latent_image = _encode_vae(vae, control_video, vae_tile_size) - concat_latent[:,:latent_channels,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] - - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent}) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - - out_latent = {} - out_latent["samples"] = latent - return (positive, negative, out_latent) - - -# ============================================================ -# XB_WanVaceToVideo — Wan VACE 视频编辑 -# ============================================================ -class XB_WanVaceToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1000.0, "step": 0.01}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "control_video": ("IMAGE",), - "control_masks": ("MASK",), - "reference_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT") - RETURN_NAMES = ("positive", "negative", "latent", "trim_latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, strength, vae_tile_size, - control_video=None, control_masks=None, reference_image=None, - scale_method="lanczos", crop_mode="center"): - latent_length = ((length - 1) // 4) + 1 - - if control_video is not None: - control_video = comfy.utils.common_upscale(control_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - if control_video.shape[0] < length: - control_video = torch.nn.functional.pad(control_video, (0, 0, 0, 0, 0, 0, 0, length - control_video.shape[0]), value=0.5) - else: - control_video = torch.ones((length, height, width, 3)) * 0.5 - - if reference_image is not None: - reference_image = comfy.utils.common_upscale(reference_image[:1].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - reference_image = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) - reference_image = torch.cat([reference_image, comfy.latent_formats.Wan21().process_out(torch.zeros_like(reference_image))], dim=1) - - if control_masks is None: - mask = torch.ones((length, height, width, 1)) - else: - mask = control_masks - if mask.ndim == 3: - mask = mask.unsqueeze(1) - mask = comfy.utils.common_upscale(mask[:length], width, height, "bilinear", "center").movedim(1, -1) - if mask.shape[0] < length: - mask = torch.nn.functional.pad(mask, (0, 0, 0, 0, 0, 0, 0, length - mask.shape[0]), value=1.0) - - control_video = control_video - 0.5 - inactive = (control_video * (1 - mask)) + 0.5 - reactive = (control_video * mask) + 0.5 - - inactive = _encode_vae(vae, inactive[:, :, :, :3], vae_tile_size) - reactive = _encode_vae(vae, reactive[:, :, :, :3], vae_tile_size) - control_video_latent = torch.cat((inactive, reactive), dim=1) - if reference_image is not None: - control_video_latent = torch.cat((reference_image, control_video_latent), dim=2) - - vae_stride = 8 - height_mask = height // vae_stride - width_mask = width // vae_stride - mask = mask.view(length, height_mask, vae_stride, width_mask, vae_stride) - mask = mask.permute(2, 4, 0, 1, 3) - mask = mask.reshape(vae_stride * vae_stride, length, height_mask, width_mask) - mask = torch.nn.functional.interpolate(mask.unsqueeze(0), size=(latent_length, height_mask, width_mask), mode='nearest-exact').squeeze(0) - - trim_latent = 0 - if reference_image is not None: - mask_pad = torch.zeros_like(mask[:, :reference_image.shape[2], :, :]) - mask = torch.cat((mask_pad, mask), dim=1) - latent_length += reference_image.shape[2] - trim_latent = reference_image.shape[2] - - mask = mask.unsqueeze(0) - - positive = node_helpers.conditioning_set_values(positive, {"vace_frames": [control_video_latent], "vace_mask": [mask], "vace_strength": [strength]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"vace_frames": [control_video_latent], "vace_mask": [mask], "vace_strength": [strength]}, append=True) - - latent = torch.zeros([batch_size, 16, latent_length, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - out_latent = {} - out_latent["samples"] = latent - return (positive, negative, out_latent, trim_latent) - - -# ============================================================ -# XB_Wan22FunControlToVideo — Wan 2.2 FunControl 转视频 -# ============================================================ -class XB_Wan22FunControlToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "ref_image": ("IMAGE",), - "start_image": ("IMAGE",), - "control_video": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, ref_image=None, start_image=None, control_video=None, scale_method="lanczos", crop_mode="center"): - spacial_scale = vae.spacial_compression_encode() - latent_channels = vae.latent_channels - latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - concat_latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) - if latent_channels == 48: - concat_latent = comfy.latent_formats.Wan22().process_out(concat_latent) - else: - concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) - concat_latent = concat_latent.repeat(1, 2, 1, 1, 1) - mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - concat_latent_image = _encode_vae(vae, start_image, vae_tile_size) - concat_latent[:,latent_channels:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] - mask[:, :, :start_image.shape[0] + 3] = 0.0 - - ref_latent = None - if ref_image is not None: - ref_image = _upscale(ref_image[:1], width, height, scale_method, crop_mode) - ref_latent = _encode_vae(vae, ref_image, vae_tile_size) - - if control_video is not None: - control_video = _upscale(control_video[:length], width, height, scale_method, crop_mode) - concat_latent_image = _encode_vae(vae, control_video, vae_tile_size) - concat_latent[:,:latent_channels,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] - - mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent, "concat_mask": mask, "concat_mask_index": latent_channels}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent, "concat_mask": mask, "concat_mask_index": latent_channels}) - - if ref_latent is not None: - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) - - out_latent = {} - out_latent["samples"] = latent - return (positive, negative, out_latent) - - -def linear_interpolation(features, input_fps, output_fps, output_len=None): - features = features.transpose(1, 2) - seq_len = features.shape[2] / float(input_fps) - if output_len is None: - output_len = int(seq_len * output_fps) - output_features = torch.nn.functional.interpolate(features, size=output_len, align_corners=True, mode='linear') - return output_features.transpose(1, 2) - -def get_sample_indices(original_fps, total_frames, target_fps, num_sample, fixed_start=None): - required_duration = num_sample / target_fps - required_origin_frames = int(np.ceil(required_duration * original_fps)) - if required_duration > total_frames / original_fps: - raise ValueError("required_duration must be less than video length") - if fixed_start is not None and fixed_start >= 0: - start_frame = fixed_start - else: - max_start = total_frames - required_origin_frames - if max_start < 0: - raise ValueError("video length is too short") - start_frame = np.random.randint(0, max_start + 1) - start_time = start_frame / original_fps - end_time = start_time + required_duration - time_points = np.linspace(start_time, end_time, num_sample, endpoint=False) - frame_indices = np.round(np.array(time_points) * original_fps).astype(int) - frame_indices = np.clip(frame_indices, 0, total_frames - 1) - return frame_indices - -def get_audio_embed_bucket_fps(audio_embed, fps=16, batch_frames=81, m=0, video_rate=30): - num_layers, audio_frame_num, audio_dim = audio_embed.shape - return_all_layers = num_layers > 1 - scale = video_rate / fps - min_batch_num = int(audio_frame_num / (batch_frames * scale)) + 1 - bucket_num = min_batch_num * batch_frames - padd_audio_num = math.ceil(min_batch_num * batch_frames / fps * video_rate) - audio_frame_num - batch_idx = get_sample_indices(video_rate, audio_frame_num + padd_audio_num, fps, bucket_num, fixed_start=0) - batch_audio_eb = [] - audio_sample_stride = int(video_rate / fps) - for bi in batch_idx: - if bi < audio_frame_num: - chosen_idx = list(range(bi - m * audio_sample_stride, bi + (m + 1) * audio_sample_stride, audio_sample_stride)) - chosen_idx = [0 if c < 0 else c for c in chosen_idx] - chosen_idx = [audio_frame_num - 1 if c >= audio_frame_num else c for c in chosen_idx] - if return_all_layers: - frame_audio_embed = audio_embed[:, chosen_idx].flatten(start_dim=-2, end_dim=-1) - else: - frame_audio_embed = audio_embed[0][chosen_idx].flatten() - else: - frame_audio_embed = torch.zeros([audio_dim * (2 * m + 1)], device=audio_embed.device) if not return_all_layers else torch.zeros([num_layers, audio_dim * (2 * m + 1)], device=audio_embed.device) - batch_audio_eb.append(frame_audio_embed) - batch_audio_eb = torch.cat([c.unsqueeze(0) for c in batch_audio_eb], dim=0) - return batch_audio_eb, min_batch_num - -def xb_wan_sound_to_video(positive, negative, vae, width, height, length, batch_size, vae_tile_size, frame_offset=0, ref_image=None, audio_encoder_output=None, control_video=None, ref_motion=None, ref_motion_latent=None, scale_method="lanczos", crop_mode="center"): - latent_t = ((length - 1) // 4) + 1 - if audio_encoder_output is not None: - feat = torch.cat(audio_encoder_output["encoded_audio_all_layers"]) - video_rate = 30 - fps = 16 - feat = linear_interpolation(feat, input_fps=50, output_fps=video_rate) - batch_frames = latent_t * 4 - audio_embed_bucket, num_repeat = get_audio_embed_bucket_fps(feat, fps=fps, batch_frames=batch_frames, m=0, video_rate=video_rate) - audio_embed_bucket = audio_embed_bucket.unsqueeze(0) - if len(audio_embed_bucket.shape) == 3: - audio_embed_bucket = audio_embed_bucket.permute(0, 2, 1) - elif len(audio_embed_bucket.shape) == 4: - audio_embed_bucket = audio_embed_bucket.permute(0, 2, 3, 1) - - audio_embed_bucket = audio_embed_bucket[:, :, :, frame_offset:frame_offset + batch_frames] - if audio_embed_bucket.shape[3] > 0: - positive = node_helpers.conditioning_set_values(positive, {"audio_embed": audio_embed_bucket}) - negative = node_helpers.conditioning_set_values(negative, {"audio_embed": audio_embed_bucket * 0.0}) - frame_offset += batch_frames - - if ref_image is not None: - ref_image = comfy.utils.common_upscale(ref_image[:1].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - ref_latent = _encode_vae(vae, ref_image, vae_tile_size) - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) - - if ref_motion is not None: - if ref_motion.shape[0] > 73: - ref_motion = ref_motion[-73:] - ref_motion = comfy.utils.common_upscale(ref_motion.movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - if ref_motion.shape[0] < 73: - r = torch.ones([73, height, width, 3]) * 0.5 - r[-ref_motion.shape[0]:] = ref_motion - ref_motion = r - ref_motion_latent = _encode_vae(vae, ref_motion, vae_tile_size) - - if ref_motion is not None: - ref_motion_latent = ref_motion_latent[:, :, -19:] - positive = node_helpers.conditioning_set_values(positive, {"reference_motion": ref_motion_latent}) - negative = node_helpers.conditioning_set_values(negative, {"reference_motion": ref_motion_latent}) - - latent = torch.zeros([batch_size, 16, latent_t, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - - control_video_out = comfy.latent_formats.Wan21().process_out(torch.zeros_like(latent)) - if control_video is not None: - control_video = comfy.utils.common_upscale(control_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - control_video = _encode_vae(vae, control_video, vae_tile_size) - control_video_out[:, :, :control_video.shape[2]] = control_video - - positive = node_helpers.conditioning_set_values(positive, {"control_video": control_video_out}) - negative = node_helpers.conditioning_set_values(negative, {"control_video": control_video_out}) - - out_latent = {} - out_latent["samples"] = latent - return positive, negative, out_latent, frame_offset - -# ============================================================ -# XB_WanSoundImageToVideo — Wan 音频+图像转视频 -# ============================================================ -class XB_WanSoundImageToVideo: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 77, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), - "ref_image": ("IMAGE",), - "control_video": ("IMAGE",), - "ref_motion": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, ref_image=None, audio_encoder_output=None, control_video=None, ref_motion=None, scale_method="lanczos", crop_mode="center"): - positive, negative, out_latent, frame_offset = xb_wan_sound_to_video( - positive, negative, vae, width, height, length, batch_size, vae_tile_size, - ref_image=ref_image, audio_encoder_output=audio_encoder_output, - control_video=control_video, ref_motion=ref_motion, - scale_method=scale_method, crop_mode=crop_mode) - return (positive, negative, out_latent) - - -from comfy.ldm.wan.model_multitalk import InfiniteTalkOuterSampleWrapper, MultiTalkCrossAttnPatch, project_audio_features - -def _linear_interp(features, input_fps, output_fps, output_len=None): - features = features.transpose(1, 2) - seq_len = features.shape[2] / float(input_fps) - if output_len is None: - output_len = int(seq_len * output_fps) - output_features = torch.nn.functional.interpolate(features, size=output_len, align_corners=True, mode='linear') - return output_features.transpose(1, 2) - -def _process_infinite_talk_audio(model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=None, start_image=None, - audio_encoder_output_2=None, previous_frames=None, - mask_1=None, mask_2=None, segment_audio=False, - global_frame_offset=None, - scale_method="lanczos", crop_mode="center"): - """共用音频处理 + 模型补丁核心逻辑（单人/双人复用）""" - - # 🌟 零拷贝克隆：所有补丁打在 clone 上，OOM/取消中断时原模型不受污染 - model = model.clone() - - if previous_frames is not None and previous_frames.shape[0] < motion_frame_count: - raise ValueError("Not enough previous frames provided.") - - if audio_encoder_output_2 is not None: - if mask_1 is None or mask_2 is None: - raise ValueError("Masks must be provided if two audio encoder outputs are used.") - - ref_masks = None - if mask_1 is not None and mask_2 is not None: - if audio_encoder_output_2 is None: - raise ValueError("Second audio encoder output must be provided if two masks are used.") - ref_masks = torch.cat([mask_1, mask_2]) - - latent = torch.zeros([1, 16, ((length - 1) // 4) + 1, height // 8, width // 8], - device=comfy.model_management.intermediate_device()) - - concat_latent_image = None - if start_image is not None: - start_image = comfy.utils.common_upscale(start_image[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) - image = torch.ones((length, height, width, start_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) * 0.5 - image[:start_image.shape[0]] = start_image - - concat_latent_image = _encode_vae(vae, image, vae_tile_size) - concat_mask = torch.ones((1, 1, latent.shape[2] * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), - device=start_image.device, dtype=start_image.dtype) - concat_mask[:, :, :start_image.shape[0] + 3] = 0.0 - concat_mask = concat_mask.view(1, concat_mask.shape[2] // 4, 4, concat_mask.shape[3], concat_mask.shape[4]).transpose(1, 2) - - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - - # === 音频处理管线 === - encoded_audio_list = [] - seq_lengths = [] - for audio_encoder_output in [audio_encoder_output_1, audio_encoder_output_2]: - if audio_encoder_output is None: - continue - all_layers = audio_encoder_output["encoded_audio_all_layers"] - encoded_audio = torch.stack(all_layers, dim=0).squeeze(1)[1:] - encoded_audio = _linear_interp(encoded_audio, input_fps=50, output_fps=25).movedim(0, 1) - encoded_audio_list.append(encoded_audio) - seq_lengths.append(encoded_audio.shape[0]) - - multi_audio_type = "add" - if len(encoded_audio_list) > 1: - if multi_audio_type == "para": - max_len = max(seq_lengths) - padded = [] - for emb in encoded_audio_list: - if emb.shape[0] < max_len: - pad = torch.zeros(max_len - emb.shape[0], *emb.shape[1:], dtype=emb.dtype) - emb = torch.cat([emb, pad], dim=0) - padded.append(emb) - encoded_audio_list = padded - elif multi_audio_type == "add": - total_len = sum(seq_lengths) - full_list = [] - offset = 0 - for emb, seq_len in zip(encoded_audio_list, seq_lengths): - full = torch.zeros(total_len, *emb.shape[1:], dtype=emb.dtype) - full[offset:offset + seq_len] = emb - full_list.append(full) - offset += seq_len - encoded_audio_list = full_list - - token_ref_target_masks = None - if ref_masks is not None: - token_ref_target_masks = torch.nn.functional.interpolate( - ref_masks.unsqueeze(0), size=(latent.shape[-2] // 2, latent.shape[-1] // 2), mode='nearest')[0] - token_ref_target_masks = (token_ref_target_masks > 0).view(token_ref_target_masks.shape[0], -1) - - if previous_frames is not None: - motion_frames = comfy.utils.common_upscale(previous_frames[-motion_frame_count:].movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1) - # 🔧 优先使用显式传入的全局偏移量，彻底废弃通过 shape 反推 - if global_frame_offset is not None: - frame_offset = global_frame_offset - else: - frame_offset = previous_frames.shape[0] - motion_frame_count - if segment_audio: - audio_start = 0 # [fixed] - else: - audio_start = frame_offset - audio_end = audio_start + length - motion_frames_latent = _encode_vae(vae, motion_frames[:, :, :, :3], vae_tile_size) - trim_image = motion_frame_count - else: - # 🔧 即使没有 previous_frames（如硬切），也要尊重全局音频偏移量 - audio_start = global_frame_offset if global_frame_offset is not None else 0 - trim_image = 0 - audio_end = audio_start + length - motion_frames_latent = concat_latent_image[:, :, :1] if concat_latent_image is not None else torch.zeros((1, 16, 1, height // 8, width // 8), device=latent.device) - - audio_embed = project_audio_features(model_patch.model.audio_proj, encoded_audio_list, audio_start, audio_end).to(model.model_dtype()) - - - for key in ("infinite_talk_outer_sample",): - if hasattr(model, "wrappers") and key in model.wrappers: - del model.wrappers[key] - for key in ("attn2_patch",): - if hasattr(model, "object_patches") and key in model.object_patches: - del model.object_patches[key] - top = model.model_options.setdefault("transformer_options", {}) - top.pop("audio_embeds", None) - - top["audio_embeds"] = audio_embed - - model.add_wrapper_with_key( - comfy.patcher_extension.WrappersMP.OUTER_SAMPLE, - "infinite_talk_outer_sample", - InfiniteTalkOuterSampleWrapper(motion_frames_latent, model_patch, is_extend=previous_frames is not None)) - model.set_model_patch(MultiTalkCrossAttnPatch(model_patch, audio_scale), "attn2_patch") - - out_latent = {"samples": latent} - return model, positive, negative, out_latent, trim_image - - -# ============================================================ -# XB_WanInfiniteTalkToVideo_Single — Wan 单人语音转视频 -# ============================================================ -class XB_WanInfiniteTalkToVideo_Single: - """单人语音转视频分块 -- 对应 WanInfiniteTalkToVideo (single_speaker) + VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model": ("MODEL",), - "model_patch": ("MODEL_PATCH",), - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), - "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), - "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "previous_frames": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") - RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=None, start_image=None, previous_frames=None, segment_audio=False, - global_frame_offset=None, - scale_method="lanczos", crop_mode="center"): - return _process_infinite_talk_audio( - model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=clip_vision_output, start_image=start_image, - previous_frames=previous_frames, segment_audio=segment_audio, - global_frame_offset=global_frame_offset) - - -# ============================================================ -# XB_WanInfiniteTalkToVideo_Dual — Wan 双人语音转视频 -# ============================================================ -class XB_WanInfiniteTalkToVideo_Dual: - """双人语音转视频分块 --对应 WanInfiniteTalkToVideo (two_speakers) + VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model": ("MODEL",), - "model_patch": ("MODEL_PATCH",), - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), - "audio_encoder_output_2": ("AUDIO_ENCODER_OUTPUT",), - "mask_1": ("MASK",), - "mask_2": ("MASK",), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), - "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), - "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "previous_frames": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") - RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, audio_encoder_output_2, mask_1, mask_2, - motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=None, start_image=None, previous_frames=None, - global_frame_offset=None, - scale_method="lanczos", crop_mode="center"): - return _process_infinite_talk_audio( - model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=clip_vision_output, start_image=start_image, - audio_encoder_output_2=audio_encoder_output_2, - previous_frames=previous_frames, - mask_1=mask_1, mask_2=mask_2, - global_frame_offset=global_frame_offset, - scale_method=scale_method, crop_mode=crop_mode) - -# ============================================================================== -# 保留旧的 XB_WanInfiniteTalkToVideo 兼容别名（单人/双人 mode 选择器） -# ============================================================================== -# ============================================================ -# XB_WanInfiniteTalkToVideo — Wan 语音转视频分块 -# ============================================================ -class XB_WanInfiniteTalkToVideo: - """兼容旧工作流 --通过 mode 切换单人/双人""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "model": ("MODEL",), - "model_patch": ("MODEL_PATCH",), - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "mode": (["single_speaker", "two_speakers"], {"default": "single_speaker"}), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), - "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), - "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), - "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "audio_encoder_output_2": ("AUDIO_ENCODER_OUTPUT",), - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "previous_frames": ("IMAGE",), - "mask_1": ("MASK",), - "mask_2": ("MASK",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") - RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, model, model_patch, positive, negative, vae, mode, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=None, start_image=None, - audio_encoder_output_2=None, previous_frames=None, - mask_1=None, mask_2=None, - global_frame_offset=None, - scale_method="lanczos", crop_mode="center"): - return _process_infinite_talk_audio( - model, model_patch, positive, negative, vae, width, height, length, - audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, - clip_vision_output=clip_vision_output, start_image=start_image, - audio_encoder_output_2=audio_encoder_output_2, - previous_frames=previous_frames, - mask_1=mask_1, mask_2=mask_2, - global_frame_offset=global_frame_offset, - scale_method=scale_method, crop_mode=crop_mode) - -# ============================================================================== - -# ============================================================================== - -# ============================================================================== - """Wan FunInpaint to Video with VAE tiling""" -# ============================================================================== -# ============================================================ -# XB_WanFunInpaintToVideo — Wan FunInpaint 转视频 -# ============================================================ -class XB_WanFunInpaintToVideo: - """Wan FunInpaint to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "end_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, - start_image=None, end_image=None, clip_vision_output=None, - scale_method="lanczos", crop_mode="center"): - flfv = XB_WanFirstLastFrameToVideo() - return flfv.process(positive, negative, vae, width, height, length, batch_size, vae_tile_size, - start_image=start_image, end_image=end_image, - clip_vision_start_image=clip_vision_output, - scale_method=scale_method, crop_mode=crop_mode) - -# ============================================================================== - """Wan Camera Control to Video with VAE tiling""" -# ============================================================================== -# ============================================================ -# XB_WanCameraImageToVideo — Wan 相机运镜图生视频 -# ============================================================ -class XB_WanCameraImageToVideo: - """Wan Camera Control to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "camera_conditions": ("WAN_CAMERA_EMBEDDING",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, - start_image=None, clip_vision_output=None, camera_conditions=None, - scale_method="lanczos", crop_mode="center"): - latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - concat_latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - concat_latent_image = _encode_vae(vae, start_image[:, :, :, :3], vae_tile_size) - concat_latent[:,:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] - mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) - mask[:, :, :start_image.shape[0] + 3] = 0.0 - mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) - positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent, "concat_mask": mask}) - negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent, "concat_mask": mask}) - - if camera_conditions is not None: - positive = node_helpers.conditioning_set_values(positive, {'camera_conditions': camera_conditions}) - negative = node_helpers.conditioning_set_values(negative, {'camera_conditions': camera_conditions}) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - - out_latent = {"samples": latent} - return (positive, negative, out_latent) - -# ============================================================================== - """Wan Phantom Subject to Video with VAE tiling""" -# ============================================================================== -# ============================================================ -# XB_WanPhantomSubjectToVideo — Wan Phantom 主体转视频 -# ============================================================ -class XB_WanPhantomSubjectToVideo: - """Wan Phantom Subject to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "images": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative_text", "negative_img_text", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, - images=None, scale_method="lanczos", crop_mode="center"): - latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - cond2 = negative - if images is not None: - images = _upscale(images[:length], width, height, scale_method, crop_mode) - latent_images = [] - for i in images: - latent_images += [_encode_vae(vae, i.unsqueeze(0)[:, :, :, :3], vae_tile_size)] - concat_latent_image = torch.cat(latent_images, dim=2) - positive = node_helpers.conditioning_set_values(positive, {"time_dim_concat": concat_latent_image}) - cond2 = node_helpers.conditioning_set_values(negative, {"time_dim_concat": concat_latent_image}) - negative = node_helpers.conditioning_set_values(negative, {"time_dim_concat": comfy.latent_formats.Wan21().process_out(torch.zeros_like(concat_latent_image))}) - - out_latent = {"samples": latent} - return (positive, cond2, negative, out_latent) - -# ============================================================================== - """Wan HuMo to Video with VAE tiling""" -# ============================================================================== -def _get_audio_emb_window(audio_emb, frame_num, frame0_idx, audio_shift=2): - zero_audio_embed = torch.zeros((audio_emb.shape[1], audio_emb.shape[2]), dtype=audio_emb.dtype, device=audio_emb.device) - zero_audio_embed_3 = torch.zeros((3, audio_emb.shape[1], audio_emb.shape[2]), dtype=audio_emb.dtype, device=audio_emb.device) - iter_ = 1 + (frame_num - 1) // 4 - audio_emb_wind = [] - for lt_i in range(iter_): - if lt_i == 0: - st = frame0_idx + lt_i - 2 - ed = frame0_idx + lt_i + 3 - wind_feat = torch.stack([audio_emb[i] if (0 <= i < audio_emb.shape[0]) else zero_audio_embed for i in range(st, ed)], dim=0) - wind_feat = torch.cat((zero_audio_embed_3, wind_feat), dim=0) - else: - st = frame0_idx + 1 + 4 * (lt_i - 1) - audio_shift - ed = frame0_idx + 1 + 4 * lt_i + audio_shift - wind_feat = torch.stack([audio_emb[i] if (0 <= i < audio_emb.shape[0]) else zero_audio_embed for i in range(st, ed)], dim=0) - audio_emb_wind.append(wind_feat) - audio_emb_wind = torch.stack(audio_emb_wind, dim=0) - return audio_emb_wind, ed - audio_shift - -# ============================================================ -# XB_WanHuMoImageToVideo — Wan HuMo 图生视频 -# ============================================================ -class XB_WanHuMoImageToVideo: - """Wan HuMo to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 97, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), - "ref_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, - ref_image=None, audio_encoder_output=None, scale_method="lanczos", crop_mode="center"): - latent_t = ((length - 1) // 4) + 1 - latent = torch.zeros([batch_size, 16, latent_t, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - - if ref_image is not None: - ref_image = _upscale(ref_image[:1], width, height, scale_method, crop_mode) - ref_latent = _encode_vae(vae, ref_image[:, :, :, :3], vae_tile_size) - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [torch.zeros_like(ref_latent)]}, append=True) - else: - zero_latent = torch.zeros([batch_size, 16, 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [zero_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [zero_latent]}, append=True) - - if audio_encoder_output is not None: - audio_emb = torch.stack(audio_encoder_output["encoded_audio_all_layers"], dim=2) - audio_len = audio_encoder_output["audio_samples"] // 640 - audio_emb = audio_emb[:, :audio_len * 2] - feat0 = linear_interpolation(audio_emb[:, :, 0: 8].mean(dim=2), 50, 25) - feat1 = linear_interpolation(audio_emb[:, :, 8: 16].mean(dim=2), 50, 25) - feat2 = linear_interpolation(audio_emb[:, :, 16: 24].mean(dim=2), 50, 25) - feat3 = linear_interpolation(audio_emb[:, :, 24: 32].mean(dim=2), 50, 25) - feat4 = linear_interpolation(audio_emb[:, :, 32], 50, 25) - audio_emb = torch.stack([feat0, feat1, feat2, feat3, feat4], dim=2)[0] - audio_emb, _ = _get_audio_emb_window(audio_emb, length, frame0_idx=0) - audio_emb = audio_emb.unsqueeze(0) - audio_emb_neg = torch.zeros_like(audio_emb) - positive = node_helpers.conditioning_set_values(positive, {"audio_embed": audio_emb}) - negative = node_helpers.conditioning_set_values(negative, {"audio_embed": audio_emb_neg}) - else: - zero_audio = torch.zeros([batch_size, latent_t + 1, 8, 5, 1280], device=comfy.model_management.intermediate_device()) - positive = node_helpers.conditioning_set_values(positive, {"audio_embed": zero_audio}) - negative = node_helpers.conditioning_set_values(negative, {"audio_embed": zero_audio}) - - out_latent = {"samples": latent} - return (positive, negative, out_latent) - -# ============================================================================== -# 7.5.5 🎬 Wan22 ImageToVideo Latent 分块 -# ============================================================================== -# ============================================================ -# XB_Wan22ImageToVideoLatent — Wan 2.2 图生视频 Latent -# ============================================================ -class XB_Wan22ImageToVideoLatent: - """Wan 2.2 Image to Video Latent with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "vae": ("VAE",), - "width": ("INT", {"default": 1280, "min": 16, "max": 8192, "step": 32}), - "height": ("INT", {"default": 704, "min": 32, "max": 8192, "step": 32}), - "length": ("INT", {"default": 49, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "start_image": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("LATENT",) - RETURN_NAMES = ("latent",) - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, vae, width, height, length, batch_size, vae_tile_size, - start_image=None, scale_method="lanczos", crop_mode="center"): - latent = torch.zeros([1, 48, ((length - 1) // 4) + 1, height // 16, width // 16], device=comfy.model_management.intermediate_device()) - mask = torch.ones([latent.shape[0], 1, ((length - 1) // 4) + 1, latent.shape[-2], latent.shape[-1]], device=comfy.model_management.intermediate_device()) - - if start_image is not None: - start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) - latent_temp = _encode_vae(vae, start_image, vae_tile_size) - latent[:, :, :latent_temp.shape[-3]] = latent_temp - mask[:, :, :latent_temp.shape[-3]] *= 0.0 - - out_latent = {} - latent_format = comfy.latent_formats.Wan22() - latent = latent_format.process_out(latent) * mask + latent * (1.0 - mask) - out_latent["samples"] = latent.repeat((batch_size,) + (1,) * (latent.ndim - 1)) - out_latent["noise_mask"] = mask.repeat((batch_size,) + (1,) * (mask.ndim - 1)) - return (out_latent,) - -# ============================================================================== -# 7.5.6 🎬 Wan SoundImage Extend 分块 -# ============================================================================== -# ============================================================ -# XB_WanSoundImageToVideoExtend — Wan 音频+图像扩展视频 -# ============================================================ -class XB_WanSoundImageToVideoExtend: - """Wan Sound Extend to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "length": ("INT", {"default": 77, "min": 1, "max": 8192, "step": 4}), - "video_latent": ("LATENT",), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), - "ref_image": ("IMAGE",), - "control_video": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, length, video_latent, vae_tile_size, - ref_image=None, audio_encoder_output=None, control_video=None, - scale_method="lanczos", crop_mode="center"): - video_latent_data = video_latent["samples"] - width = video_latent_data.shape[-1] * 8 - height = video_latent_data.shape[-2] * 8 - batch_size = video_latent_data.shape[0] - frame_offset = video_latent_data.shape[-3] * 4 - positive, negative, out_latent, frame_offset = xb_wan_sound_to_video( - positive, negative, vae, width, height, length, batch_size, vae_tile_size, - frame_offset=frame_offset, ref_image=ref_image, - audio_encoder_output=audio_encoder_output, - control_video=control_video, ref_motion=None, ref_motion_latent=video_latent_data, - scale_method=scale_method, crop_mode=crop_mode) - return (positive, negative, out_latent) - -# ============================================================================== - """Wan SCAIL Pose to Video with VAE tiling""" -# ============================================================================== -# ============================================================ -# XB_WanSCAILToVideo — Wan SCAIL 转视频 -# ============================================================ -class XB_WanSCAILToVideo: - """Wan SCAIL Pose to Video with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), - "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), - "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "reference_image": ("IMAGE",), - "pose_video": ("IMAGE",), - "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), - "crop_mode": (_CROP_MODES, {"default": "center"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, pose_strength, pose_start, pose_end, vae_tile_size, - reference_image=None, clip_vision_output=None, pose_video=None, - scale_method="lanczos", crop_mode="center"): - latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - - if reference_image is not None: - reference_image = _upscale(reference_image[:1], width, height, scale_method, crop_mode) - ref_latent = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [torch.zeros_like(ref_latent)]}, append=True) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - - if pose_video is not None: - pose_video = _upscale(pose_video[:length], width // 2, height // 2, "area", "center") - pose_video_latent = _encode_vae(vae, pose_video[:, :, :, :3], vae_tile_size) * pose_strength - positive = node_helpers.conditioning_set_values_with_timestep_range(positive, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) - negative = node_helpers.conditioning_set_values_with_timestep_range(negative, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) - - out_latent = {"samples": latent} - return (positive, negative, out_latent) - -# ============================================================================== -# 7.5.7 Pro 🎬 Wan SCAIL (pose) 转视频分块 — 高配版 -# ============================================================================== -# SCAIL-2 mask helper — colored RGB mask → 28-channel binary latent -# ============================================================================== -def _scail_extract_mask_to_28ch(rgb_video): - """Colored RGB mask (T, H, W, 3) in [0,1] -> SCAIL-2 28-channel binary latent - (1, T_lat, 28, H_lat, W_lat). 7 per-color binary channels (white/r/g/b/y/m/c) - threshold-extracted at 225/255, 8x spatial downsample, 4-frame temporal stacking.""" - T, H, W, _ = rgb_video.shape - _ON_THRESH = 225.0 / 255.0 - mask = rgb_video.movedim(-1, 1).float() - R = (mask[:, 0:1] > _ON_THRESH).float() - G = (mask[:, 1:2] > _ON_THRESH).float() - B = (mask[:, 2:3] > _ON_THRESH).float() - nR, nG, nB = 1 - R, 1 - G, 1 - B - binary_7ch = torch.cat([ - R * G * B, # white - R * nG * nB, # red - nR * G * nB, # green - nR * nG * B, # blue - R * G * nB, # yellow - R * nG * B, # magenta - nR * G * B, # cyan - ], dim=1) - H_lat, W_lat = H, W - for _ in range(3): - H_lat = (H_lat + 1) // 2 - W_lat = (W_lat + 1) // 2 - binary_7ch = torch.nn.functional.interpolate(binary_7ch, size=(H_lat, W_lat), mode='area') - T_latent = (T - 1) // 4 + 1 - padded = torch.cat([binary_7ch[:1].repeat(4, 1, 1, 1), binary_7ch[1:]], dim=0) - out = padded.view(T_latent, 28, H_lat, W_lat) - return out.unsqueeze(0) - -# ============================================================================== -# 7.5.7 Pro 🎬 Wan SCAIL-2 (pose) 转视频分块 — 完整版 -# ============================================================================== -# ============================================================ -# XB_WanSCAILToVideoPro — Wan SCAIL Pro 转视频 -# ============================================================ -class XB_WanSCAILToVideoPro: - """Wan SCAIL-2 Pose to Video Pro — exact port from ComfyUI core nodes_scail.py with VAE tiling""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), - "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), - "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), - "replacement_mode": ("BOOLEAN", {"default": False}), - "video_frame_offset": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}), - "previous_frame_count": ("INT", {"default": 5, "min": 1, "max": 8192, "step": 4}), - "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "pose_video": ("IMAGE",), - "pose_video_mask": ("IMAGE",), - "reference_image": ("IMAGE",), - "reference_image_mask": ("IMAGE",), - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "previous_frames": ("IMAGE",), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT") - RETURN_NAMES = ("positive", "negative", "latent", "video_frame_offset") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, batch_size, pose_strength, pose_start, pose_end, - replacement_mode, video_frame_offset, previous_frame_count, vae_tile_size, - pose_video=None, pose_video_mask=None, reference_image=None, - reference_image_mask=None, clip_vision_output=None, previous_frames=None): - latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) - noise_mask = None - - # ref_mask_flag: False=Animation Mode, True=Replacement Mode - ref_mask_flag = not replacement_mode - positive = node_helpers.conditioning_set_values(positive, {"ref_mask_flag": ref_mask_flag}) - negative = node_helpers.conditioning_set_values(negative, {"ref_mask_flag": ref_mask_flag}) - - # --- previous_frames --- - prev_trimmed = None - if previous_frames is not None and previous_frames.shape[0] > 0: - prev_trimmed = previous_frames[-previous_frame_count:] - video_frame_offset -= prev_trimmed.shape[0] - video_frame_offset = max(0, video_frame_offset) - - # --- reference_image + mask --- - if reference_image is not None: - reference_image = comfy.utils.common_upscale(reference_image[:1].movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) - # Replacement Mode: composite ref on black bg using reference_image_mask as alpha matte - if replacement_mode and reference_image_mask is not None: - rm = comfy.utils.common_upscale(reference_image_mask[:1].movedim(-1, 1), width, height, "nearest-exact", "center").movedim(1, -1) - is_char = (rm[..., :3].max(dim=-1, keepdim=True).values > 0.1).to(reference_image.dtype) - reference_image = reference_image * is_char - ref_latent = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) - positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) - negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) - negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) - - # --- pose_video (with offset truncation) --- - if pose_video is not None: - if pose_video.shape[0] <= video_frame_offset: - pose_video = None - else: - pose_video = pose_video[video_frame_offset:] - if pose_video_mask is not None: - if pose_video_mask.shape[0] <= video_frame_offset: - pose_video_mask = None - else: - pose_video_mask = pose_video_mask[video_frame_offset:] - - # Truncate pose+mask jointly to the shorter of the two, capped at length - ts = [v.shape[0] for v in (pose_video, pose_video_mask) if v is not None] - if ts: - T_kept = ((min(min(ts), length) - 1) // 4) * 4 + 1 - if pose_video is not None: - pose_video = pose_video[:T_kept] - if pose_video_mask is not None: - pose_video_mask = pose_video_mask[:T_kept] - - if pose_video is not None: - pose_video = comfy.utils.common_upscale(pose_video[:length].movedim(-1, 1), width // 2, height // 2, "area", "center").movedim(1, -1) - pose_video_latent = _encode_vae(vae, pose_video[:, :, :, :3], vae_tile_size) * pose_strength - positive = node_helpers.conditioning_set_values_with_timestep_range(positive, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) - negative = node_helpers.conditioning_set_values_with_timestep_range(negative, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) - - if pose_video_mask is not None: - mask_video_hw = comfy.utils.common_upscale(pose_video_mask[:length].movedim(-1, 1), width // 2, height // 2, "area", "center").movedim(1, -1) - driving_mask_28ch = _scail_extract_mask_to_28ch(mask_video_hw) - positive = node_helpers.conditioning_set_values(positive, {"driving_mask_28ch": driving_mask_28ch}) - negative = node_helpers.conditioning_set_values(negative, {"driving_mask_28ch": driving_mask_28ch}) - - if reference_image_mask is not None and replacement_mode: - ref_mask_hw = comfy.utils.common_upscale(reference_image_mask[:1].movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) - ref_mask_1f = _scail_extract_mask_to_28ch(ref_mask_hw) - zeros = torch.zeros((1, latent.shape[2], 28, ref_mask_1f.shape[-2], ref_mask_1f.shape[-1]), device=ref_mask_1f.device, dtype=ref_mask_1f.dtype) - ref_mask_28ch = torch.cat([ref_mask_1f, zeros], dim=1) - positive = node_helpers.conditioning_set_values(positive, {"ref_mask_28ch": ref_mask_28ch}) - negative = node_helpers.conditioning_set_values(negative, {"ref_mask_28ch": ref_mask_28ch}) - - if prev_trimmed is not None: - pf = comfy.utils.common_upscale(prev_trimmed.movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) - prev_latent = _encode_vae(vae, pf[:, :, :, :3], vae_tile_size) - prev_latent_frames = min(prev_latent.shape[2], latent.shape[2]) - latent[:, :, :prev_latent_frames] = prev_latent[:, :, :prev_latent_frames].to(latent.dtype) - noise_mask = torch.ones((1, 1, latent.shape[2], latent.shape[-2], latent.shape[-1]), device=latent.device, dtype=latent.dtype) - noise_mask[:, :, :prev_latent_frames] = 0.0 - - out_latent = {"samples": latent} - if noise_mask is not None: - out_latent["noise_mask"] = noise_mask - return (positive, negative, out_latent, video_frame_offset + length) - -# ============================================================================== -# 7.6 🎬 WanDancerVideo — Wan Dancer 音视频生成 (VAE 分块版) -# ============================================================================== -# ============================================================ -# XB_WanDancerVideo — Wan Dancer 转视频 (VAE 分块) -# ============================================================ -class XB_WanDancerVideo: - """Wan Dancer Video with VAE tiling —— 完全复刻官方 WanDancerVideo 逻辑， - 仅将 vae.encode() 替换为 _encode_vae() 以支持空间/时间分块。""" - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 149, "min": 1, "max": 8192, "step": 4, "tooltip": "生成视频的总帧数"}), - "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - }, - "optional": { - "clip_vision_output": ("CLIP_VISION_OUTPUT",), - "clip_vision_output_ref": ("CLIP_VISION_OUTPUT",), - "start_image": ("IMAGE",), - "mask": ("MASK",), - "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Pipeline" - - def process(self, positive, negative, vae, width, height, length, vae_tile_size, - start_image=None, mask=None, - clip_vision_output=None, clip_vision_output_ref=None, - audio_encoder_output=None): - latent = torch.zeros([1, 16, ((length - 1) // 4) + 1, height // 8, width // 8], - device=comfy.model_management.intermediate_device()) - - if start_image is not None: - start_image = comfy.utils.common_upscale( - start_image[:length].movedim(-1, 1), width, height, "bilinear", "center" - ).movedim(1, -1) - image = torch.zeros((length, height, width, start_image.shape[-1]), - device=start_image.device, dtype=start_image.dtype) - image[:start_image.shape[0]] = start_image - - # 🔧 核心改动：使用 _encode_vae 替代 vae.encode()，支持空间/时间分块 - concat_latent_image = _encode_vae(vae, image[:, :, :, :3], vae_tile_size) - - if mask is None: - concat_mask = torch.ones( - (1, 1, latent.shape[2], concat_latent_image.shape[-2], concat_latent_image.shape[-1]), - device=start_image.device, dtype=start_image.dtype) - concat_mask[:, :, :((start_image.shape[0] - 1) // 4) + 1] = 0.0 - else: - concat_mask = 1 - mask[:length].unsqueeze(0) - concat_mask = comfy.utils.common_upscale( - concat_mask, concat_latent_image.shape[-2], concat_latent_image.shape[-1], - "nearest-exact", "disabled") - concat_mask = torch.cat( - [torch.repeat_interleave(concat_mask[:, 0:1], repeats=4, dim=1), concat_mask[:, 1:]], dim=1) - concat_mask = concat_mask.view( - 1, concat_mask.shape[1] // 4, 4, - concat_latent_image.shape[-2], concat_latent_image.shape[-1] - ).transpose(1, 2) - - positive = node_helpers.conditioning_set_values( - positive, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) - negative = node_helpers.conditioning_set_values( - negative, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) - - if clip_vision_output is not None: - positive = node_helpers.conditioning_set_values( - positive, {"clip_vision_output": clip_vision_output, "clip_vision_output_ref": clip_vision_output_ref}) - negative = node_helpers.conditioning_set_values( - negative, {"clip_vision_output": clip_vision_output, "clip_vision_output_ref": clip_vision_output_ref}) - - if audio_encoder_output is not None: - positive = node_helpers.conditioning_set_values(positive, { - "audio_embed": audio_encoder_output["audio_feature"], - "fps": audio_encoder_output["fps"], - "audio_inject_scale": audio_encoder_output.get("audio_inject_scale", 1.0) - }) - negative = node_helpers.conditioning_set_values(negative, { - "audio_embed": audio_encoder_output["audio_feature"], - "fps": audio_encoder_output["fps"], - "audio_inject_scale": audio_encoder_output.get("audio_inject_scale", 1.0) - }) - - out_latent = {"samples": latent} - return (positive, negative, out_latent) - -# ============================================================ -# XB_WanDanceSwitcher — Wan 舞蹈切换器 -# ============================================================ -class XB_WanDanceSwitcher: - """Wan 舞蹈切换器 — 选择舞蹈类型和动作幅度，输出对应提示词。 - - 替换原工作流中 CustomCombo + RegexExtract + StringReplace + StringConcatenate 节点组， - 简化为一键切换。输出与原版完全一致的中文提示词。 - """ - _DANCE_STYLES = [ - "古典舞", - "韩舞", - "街舞", - "拉丁舞", - "踢踏舞", - ] - _AMPLITUDES = [ - "低", - "中等", - "高", - "最大", - ] - - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "dance_style": (cls._DANCE_STYLES, {"default": "古典舞"}), - "motion_amplitude": (cls._AMPLITUDES, {"default": "中等"}), - } - } - - RETURN_TYPES = ("STRING", "STRING") - RETURN_NAMES = ("舞蹈风格提示词", "整体描述提示词") - FUNCTION = "switch" - CATEGORY = "XB_ToolBox/Wan" - - def switch(self, dance_style, motion_amplitude): - # 输出1 (接18号节点): 舞蹈风格名称（原版 RegexExtract 输出） - # 输出2 (接20号节点): 整体描述提示词（原版 StringConcatenate 输出） - dance_prompt = f"一个人正在跳舞，舞蹈种类是{dance_style}" - full_prompt = f"{dance_prompt},图像清晰程度高,人物动作幅度{motion_amplitude}" - - return (dance_style, full_prompt) - -# ============================================================ -# XB_WanDancerCombo — Wan Dancer 综合输入 (音频切片 + 特征提取 + 舞蹈选择) -# ============================================================ -import os as _os -import folder_paths as _folder_paths - -class XB_WanDancerCombo: - """Wan Dancer 综合输入节点 —— 融合音频切片 + 音频特征提取 + 舞蹈切换器。 - - 界面照抄 AudioSlicerV1，内部集成了 WanDancerEncodeAudio 的全套音频特征提取 - 和 XB_WanDanceSwitcher 的舞蹈类型/动作幅度选择。 - - 输入: 音频文件 + 起止时间 + 视频帧数 + 注入比例 + 舞蹈类型 + 动作幅度 - 输出: 切片音频 + 舞蹈风格提示词(含FPS) + 整体描述提示词 - """ - _DANCE_STYLES = ["古典舞", "韩舞", "街舞", "拉丁舞", "踢踏舞"] - _AMPLITUDES = ["低", "中等", "高", "最大"] - - @classmethod - def INPUT_TYPES(cls): - input_dir = _folder_paths.get_input_directory() - audio_files = ["none"] - if _os.path.exists(input_dir): - for f in sorted(_os.listdir(input_dir)): - if f.lower().endswith(('.mp3','.wav','.flac','.ogg','.m4a','.aac','.opus','.wma','.webm')): - audio_files.append(f) - - return { - "required": { - "audio": (audio_files,), - "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), - "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), - "video_frames": ("INT", {"default": 149, "min": 1, "max": 8192, "step": 4, "tooltip": "视频总帧数"}), - "audio_inject_scale": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01, "tooltip": "音频注入强度"}), - "dance_style": (cls._DANCE_STYLES, {"default": "古典舞"}), - "motion_amplitude": (cls._AMPLITUDES, {"default": "中等"}), - } - } - - RETURN_TYPES = ("AUDIO", "STRING", "STRING") - RETURN_NAMES = ("音频", "整体描述提示词", "舞蹈风格提示词") - FUNCTION = "process" - CATEGORY = "XB_ToolBox/Wan" - - @classmethod - def VALIDATE_INPUTS(cls, audio, **kwargs): - return True - - def process(self, audio, start_time, end_time, duration_display, - video_frames, audio_inject_scale, dance_style, motion_amplitude): - # ── 1. 音频切片 (照抄 AudioSlicerV1) ── - from .nodes_audio_slicer import _load_audio_file, _make_audio, _snap_4n1 - - if audio == "none": - waveform = torch.zeros((1, 1), dtype=torch.float32) - sample_rate = 44100 - else: - result = _load_audio_file(_os.path.join(_folder_paths.get_input_directory(), audio)) - if result is None: - waveform = torch.zeros((1, 1), dtype=torch.float32) - sample_rate = 44100 - else: - waveform, sample_rate = result - - if waveform.shape[0] > 1: - waveform = waveform.mean(dim=0, keepdim=True) - - total_duration = waveform.shape[1] / sample_rate - start_time = max(0.0, min(start_time, total_duration)) - end_time = max(start_time, min(end_time, total_duration)) - - start_sample = int(start_time * sample_rate) - end_sample = int(end_time * sample_rate) - if end_sample <= start_sample: - end_sample = min(start_sample + int(0.1 * sample_rate), waveform.shape[1]) - - sliced_waveform = waveform[:, start_sample:end_sample] - - # ── 2. 音频特征提取 (照抄 WanDancerEncodeAudio) ── - try: - import torchaudio as _torchaudio - except ImportError: - raise ImportError( - "\n❌ XB_WanDancerCombo 需要 torchaudio 库。\n" - " 请运行: pip install torchaudio" - ) - try: - from comfy_extras.nodes_wandancer import ( - quick_tempo_estimate, estimate_tempo_from_onset, - _compute_mel_spectrogram, power_to_db, compute_onset_envelope, - compute_mfcc, compute_chroma_cens, detect_onset_peaks, - track_beats - ) - except ImportError as e: - raise ImportError( - "\n❌ XB_WanDancerCombo 需要 ComfyUI 内置的 WanDancer 模块\n" - " (comfy_extras.nodes_wandancer)，但当前版本未找到。\n" - " 请升级 ComfyUI 到 0.28.0 或更新版本！\n" - f" 原始错误: {e}" - ) - - try: - import scipy.signal, scipy.ndimage, scipy.fft, scipy.sparse # noqa: F401 WanDancer 特征提取依赖 - except ImportError: - raise ImportError( - "\n❌ XB_WanDancerCombo 需要 scipy 库。\n" - " 请运行: pip install scipy" - ) - - wf_enc = sliced_waveform.squeeze() - sr_enc = sample_rate - base_fps = 30 - hop_length = 512 - model_sr = 22050 - n_fft = 2048 - - start_bpm = quick_tempo_estimate(wf_enc.cpu().numpy(), sr_enc, hop_length=hop_length) - - resample_sr = base_fps * hop_length - wf_enc = _torchaudio.functional.resample(wf_enc.unsqueeze(0), sr_enc, resample_sr).squeeze(0) - - waveform_np = wf_enc.cpu().numpy() - mel_spec = _compute_mel_spectrogram(waveform_np, model_sr, n_fft, hop_length, n_mels=128) - mel_spec_db = power_to_db(mel_spec, amin=1e-10, top_db=80.0, ref=1.0) - envelope = compute_onset_envelope(mel_spec_db, n_fft, hop_length) - mfcc = compute_mfcc(mel_spec_db, n_mfcc=20) - chroma = compute_chroma_cens(y=waveform_np, sr=model_sr, hop_length=hop_length).T - peak_idxs = detect_onset_peaks(envelope, sr=model_sr, hop_length=hop_length) - peak_onehot = np.zeros_like(envelope, dtype=np.float32) - peak_onehot[peak_idxs] = 1.0 - beat_tracking_tempo = estimate_tempo_from_onset(envelope, sr=model_sr, hop_length=hop_length, start_bpm=start_bpm) - beat_idxs = track_beats(envelope, beat_tracking_tempo, model_sr, hop_length, tightness=100, trim=True) - beat_onehot = np.zeros_like(envelope, dtype=np.float32) - beat_onehot[beat_idxs] = 1.0 - - audio_feature = np.concatenate( - [envelope[:, None], mfcc, chroma, peak_onehot[:, None], beat_onehot[:, None]], axis=-1) - audio_feature = torch.from_numpy(audio_feature).unsqueeze(0).to(comfy.model_management.intermediate_device()) - - # fps 基于切片后音频的特征长度计算（与原版 WanDancerEncodeAudio 逻辑完全一致） - fps = float(base_fps / int(audio_feature.shape[1] / video_frames + 0.5)) - if int(fps + 0.5) != 30: - fps_string = " 帧率是{:.4f}".format(fps) - else: - fps_string = ", 帧率是30fps。" - - # ── 3. 构建音频输出 (含 audio_encoder_output 特征) ── - audio_out = { - "waveform": sliced_waveform.unsqueeze(0).unsqueeze(0), - "sample_rate": sample_rate, - "audio_feature": audio_feature, - "fps": fps, - "audio_inject_scale": audio_inject_scale, - } - - # ── 4. 提示词生成 (照抄 WanDanceSwitcher) ── - dance_prompt = f"{dance_style}{fps_string}" - full_prompt = f"一个人正在跳舞，舞蹈种类是{dance_style},图像清晰程度高,人物动作幅度{motion_amplitude}" - - return (audio_out, full_prompt, dance_prompt) - -# ============================================================ -# XB_BerniniConditioning - Bernini VAE分块编码 -# ============================================================ - -def _resize_long_edge(image, max_size, stride=16, method="area"): - h, w = image.shape[1], image.shape[2] - scale = min(max_size / max(h, w), 1.0) - nh = max(stride, round(h * scale / stride) * stride) - nw = max(stride, round(w * scale / stride) * stride) - return comfy.utils.common_upscale(image[:, :, :, :3].movedim(-1, 1), nw, nh, method, "disabled").movedim(1, -1) - - -class XB_BerniniConditioning: - @classmethod - def INPUT_TYPES(cls): - return { - "required": { - "positive": ("CONDITIONING",), - "negative": ("CONDITIONING",), - "vae": ("VAE",), - "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), - "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), - "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), - "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), - }, - "optional": { - "source_video": ("IMAGE",), - "reference_video": ("IMAGE",), - "reference_image_0": ("IMAGE",), - "reference_image_1": ("IMAGE",), - "reference_image_2": ("IMAGE",), - "reference_image_3": ("IMAGE",), - "reference_image_4": ("IMAGE",), - "reference_image_5": ("IMAGE",), - "reference_image_6": ("IMAGE",), - "reference_image_7": ("IMAGE",), - "ref_max_size": ("INT", {"default": 848, "min": 16, "max": 8192, "step": 16}), - "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), - "scale_method": (_SCALE_METHODS, {"default": "area"}), - } - } - - RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") - RETURN_NAMES = ("positive", "negative", "latent") - FUNCTION = "go" - CATEGORY = "XB_ToolBox/Wan" - - def go(self, positive, negative, vae, width, height, length, batch_size, - source_video=None, reference_video=None, - reference_image_0=None, reference_image_1=None, - reference_image_2=None, reference_image_3=None, - reference_image_4=None, reference_image_5=None, - reference_image_6=None, reference_image_7=None, - ref_max_size=848, vae_tile_size=256, scale_method="area"): - latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], - device=comfy.model_management.intermediate_device()) - context = [] - if source_video is not None: - vid = comfy.utils.common_upscale(source_video[:length, :, :, :3].movedim(-1, 1), width, height, scale_method, "center").movedim(1, -1) - context.append(_encode_vae(vae, vid[:, :, :, :3], vae_tile_size)) - if reference_video is not None: - ref_vid = _resize_long_edge(reference_video[:length], ref_max_size, method=scale_method) - context.append(_encode_vae(vae, ref_vid[:, :, :, :3], vae_tile_size)) - ref_imgs = [reference_image_0,reference_image_1,reference_image_2,reference_image_3,reference_image_4,reference_image_5,reference_image_6,reference_image_7] - for imgs in ref_imgs: - if imgs is None: continue - for i in range(imgs.shape[0]): - img = _resize_long_edge(imgs[i:i+1], ref_max_size, method=scale_method) - context.append(_encode_vae(vae, img[:,:,:,:3], vae_tile_size)) - if context: - positive = node_helpers.conditioning_set_values(positive, {"context_latents": context}) - negative = node_helpers.conditioning_set_values(negative, {"context_latents": context}) +import torch +import comfy.model_management +import comfy.utils +import comfy.patcher_extension +import node_helpers +import comfy.clip_vision +import math +import numpy as np +import nodes + +_SCALE_METHODS = ["lanczos", "bilinear", "bicubic", "nearest-exact", "area"] +_CROP_MODES = ["center", "disabled"] + +def _upscale(img, w, h, method="lanczos", crop="center"): + """Upscale helper for all nodes""" + return comfy.utils.common_upscale(img.movedim(-1, 1), w, h, method, crop).movedim(1, -1) + +def _encode_vae(vae, pixels, tile_size): + if pixels.dim() == 3: + pixels = pixels.unsqueeze(0) + p = pixels[:, :, :, :3] + + # 🛡️ 显存连续性重组 (movedim/切片会产生非连续张量，MIOpen 无法处理) + if not p.is_contiguous(): + p = p.contiguous() + + # 🛡️ VAE 精度对齐 (输入 float32 vs 权重 bfloat16 会 RuntimeError) + if hasattr(vae, 'first_stage_model'): + vae_dtype = getattr(vae.first_stage_model, 'dtype', None) + if vae_dtype is None: + try: vae_dtype = next(vae.first_stage_model.parameters()).dtype + except: pass + if vae_dtype is not None and p.dtype != vae_dtype: + p = p.to(vae_dtype) + + if tile_size == 0 or tile_size is None: + return vae.encode(p) + else: + # 🔧 overlap_t = tile_t//2: Wan 3D VAE 时间卷积需要充足上下文帧， + # overlap_t=4 (12.5%) 导致分块边界 latent 衰减 → 解码尾帧发灰 + return vae.encode_tiled(p, tile_x=tile_size, tile_y=tile_size, overlap=32, tile_t=32, overlap_t=16) + +# ============================================================ +# XB_WanImageToVideo — Wan 图生视频 +# ============================================================ +class XB_WanImageToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, clip_vision_output=None, start_image=None, scale_method="lanczos", crop_mode="center"): + spacial_scale = vae.spacial_compression_encode() + latent_channels = vae.latent_channels + latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + image = torch.ones((length, height, width, start_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) * 0.5 + image[:start_image.shape[0]] = start_image + + concat_latent_image = _encode_vae(vae, image, vae_tile_size) + + mask = torch.ones((1, 1, latent.shape[2] * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) + mask[:, :, :start_image.shape[0] + 3] = 0.0 + mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) + + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + + return (positive, negative, {"samples": latent}) + + +# ============================================================ +# XB_WanFirstLastFrameToVideo — Wan 首尾帧转视频 +# ============================================================ +class XB_WanFirstLastFrameToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 1}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 8192, "step": 1}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_start_image": ("CLIP_VISION_OUTPUT",), + "clip_vision_end_image": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "end_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, clip_vision_start_image=None, clip_vision_end_image=None, start_image=None, end_image=None, scale_method="lanczos", crop_mode="center"): + spacial_scale = vae.spacial_compression_encode() + latent = torch.zeros([batch_size, vae.latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + if end_image is not None: + end_image = _upscale(end_image[-length:], width, height, scale_method, crop_mode) + + image = torch.ones((length, height, width, 3), device=latent.device) * 0.5 + mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) + + if start_image is not None: + image[:start_image.shape[0]] = start_image + mask[:, :, :start_image.shape[0] + 3] = 0.0 + + if end_image is not None: + image[-end_image.shape[0]:] = end_image + mask[:, :, -end_image.shape[0]:] = 0.0 + + concat_latent_image = _encode_vae(vae, image, vae_tile_size) + + mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": mask}) + + clip_vision_output = None + if clip_vision_start_image is not None: + clip_vision_output = clip_vision_start_image + + if clip_vision_end_image is not None: + if clip_vision_output is not None: + states = torch.cat([clip_vision_output.penultimate_hidden_states, clip_vision_end_image.penultimate_hidden_states], dim=-2) + clip_vision_output = comfy.clip_vision.Output() + clip_vision_output.penultimate_hidden_states = states + else: + clip_vision_output = clip_vision_end_image + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + + return (positive, negative, {"samples": latent}) + + +# ============================================================ +# XB_WanFunControlToVideo — Wan FunControl 转视频 +# ============================================================ +class XB_WanFunControlToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "control_video": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, start_image=None, clip_vision_output=None, control_video=None, scale_method="lanczos", crop_mode="center"): + spacial_scale = vae.spacial_compression_encode() + latent_channels = vae.latent_channels + latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + concat_latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) + concat_latent = concat_latent.repeat(1, 2, 1, 1, 1) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + concat_latent_image = _encode_vae(vae, start_image, vae_tile_size) + concat_latent[:,latent_channels:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] + + if control_video is not None: + control_video = _upscale(control_video[:length], width, height, scale_method, crop_mode) + concat_latent_image = _encode_vae(vae, control_video, vae_tile_size) + concat_latent[:,:latent_channels,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] + + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent}) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + + out_latent = {} + out_latent["samples"] = latent + return (positive, negative, out_latent) + + +# ============================================================ +# XB_WanVaceToVideo — Wan VACE 视频编辑 +# ============================================================ +class XB_WanVaceToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1000.0, "step": 0.01}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "control_video": ("IMAGE",), + "control_masks": ("MASK",), + "reference_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT") + RETURN_NAMES = ("positive", "negative", "latent", "trim_latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, strength, vae_tile_size, + control_video=None, control_masks=None, reference_image=None, + scale_method="lanczos", crop_mode="center"): + latent_length = ((length - 1) // 4) + 1 + + if control_video is not None: + control_video = comfy.utils.common_upscale(control_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + if control_video.shape[0] < length: + control_video = torch.nn.functional.pad(control_video, (0, 0, 0, 0, 0, 0, 0, length - control_video.shape[0]), value=0.5) + else: + control_video = torch.ones((length, height, width, 3)) * 0.5 + + if reference_image is not None: + reference_image = comfy.utils.common_upscale(reference_image[:1].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + reference_image = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) + reference_image = torch.cat([reference_image, comfy.latent_formats.Wan21().process_out(torch.zeros_like(reference_image))], dim=1) + + if control_masks is None: + mask = torch.ones((length, height, width, 1)) + else: + mask = control_masks + if mask.ndim == 3: + mask = mask.unsqueeze(1) + mask = comfy.utils.common_upscale(mask[:length], width, height, "bilinear", "center").movedim(1, -1) + if mask.shape[0] < length: + mask = torch.nn.functional.pad(mask, (0, 0, 0, 0, 0, 0, 0, length - mask.shape[0]), value=1.0) + + control_video = control_video - 0.5 + inactive = (control_video * (1 - mask)) + 0.5 + reactive = (control_video * mask) + 0.5 + + inactive = _encode_vae(vae, inactive[:, :, :, :3], vae_tile_size) + reactive = _encode_vae(vae, reactive[:, :, :, :3], vae_tile_size) + control_video_latent = torch.cat((inactive, reactive), dim=1) + if reference_image is not None: + control_video_latent = torch.cat((reference_image, control_video_latent), dim=2) + + vae_stride = 8 + height_mask = height // vae_stride + width_mask = width // vae_stride + mask = mask.view(length, height_mask, vae_stride, width_mask, vae_stride) + mask = mask.permute(2, 4, 0, 1, 3) + mask = mask.reshape(vae_stride * vae_stride, length, height_mask, width_mask) + mask = torch.nn.functional.interpolate(mask.unsqueeze(0), size=(latent_length, height_mask, width_mask), mode='nearest-exact').squeeze(0) + + trim_latent = 0 + if reference_image is not None: + mask_pad = torch.zeros_like(mask[:, :reference_image.shape[2], :, :]) + mask = torch.cat((mask_pad, mask), dim=1) + latent_length += reference_image.shape[2] + trim_latent = reference_image.shape[2] + + mask = mask.unsqueeze(0) + + positive = node_helpers.conditioning_set_values(positive, {"vace_frames": [control_video_latent], "vace_mask": [mask], "vace_strength": [strength]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"vace_frames": [control_video_latent], "vace_mask": [mask], "vace_strength": [strength]}, append=True) + + latent = torch.zeros([batch_size, 16, latent_length, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + out_latent = {} + out_latent["samples"] = latent + return (positive, negative, out_latent, trim_latent) + + +# ============================================================ +# XB_Wan22FunControlToVideo — Wan 2.2 FunControl 转视频 +# ============================================================ +class XB_Wan22FunControlToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "ref_image": ("IMAGE",), + "start_image": ("IMAGE",), + "control_video": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, ref_image=None, start_image=None, control_video=None, scale_method="lanczos", crop_mode="center"): + spacial_scale = vae.spacial_compression_encode() + latent_channels = vae.latent_channels + latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + concat_latent = torch.zeros([batch_size, latent_channels, ((length - 1) // 4) + 1, height // spacial_scale, width // spacial_scale], device=comfy.model_management.intermediate_device()) + if latent_channels == 48: + concat_latent = comfy.latent_formats.Wan22().process_out(concat_latent) + else: + concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) + concat_latent = concat_latent.repeat(1, 2, 1, 1, 1) + mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + concat_latent_image = _encode_vae(vae, start_image, vae_tile_size) + concat_latent[:,latent_channels:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] + mask[:, :, :start_image.shape[0] + 3] = 0.0 + + ref_latent = None + if ref_image is not None: + ref_image = _upscale(ref_image[:1], width, height, scale_method, crop_mode) + ref_latent = _encode_vae(vae, ref_image, vae_tile_size) + + if control_video is not None: + control_video = _upscale(control_video[:length], width, height, scale_method, crop_mode) + concat_latent_image = _encode_vae(vae, control_video, vae_tile_size) + concat_latent[:,:latent_channels,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] + + mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent, "concat_mask": mask, "concat_mask_index": latent_channels}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent, "concat_mask": mask, "concat_mask_index": latent_channels}) + + if ref_latent is not None: + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) + + out_latent = {} + out_latent["samples"] = latent + return (positive, negative, out_latent) + + +def linear_interpolation(features, input_fps, output_fps, output_len=None): + features = features.transpose(1, 2) + seq_len = features.shape[2] / float(input_fps) + if output_len is None: + output_len = int(seq_len * output_fps) + output_features = torch.nn.functional.interpolate(features, size=output_len, align_corners=True, mode='linear') + return output_features.transpose(1, 2) + +def get_sample_indices(original_fps, total_frames, target_fps, num_sample, fixed_start=None): + required_duration = num_sample / target_fps + required_origin_frames = int(np.ceil(required_duration * original_fps)) + if required_duration > total_frames / original_fps: + raise ValueError("required_duration must be less than video length") + if fixed_start is not None and fixed_start >= 0: + start_frame = fixed_start + else: + max_start = total_frames - required_origin_frames + if max_start < 0: + raise ValueError("video length is too short") + start_frame = np.random.randint(0, max_start + 1) + start_time = start_frame / original_fps + end_time = start_time + required_duration + time_points = np.linspace(start_time, end_time, num_sample, endpoint=False) + frame_indices = np.round(np.array(time_points) * original_fps).astype(int) + frame_indices = np.clip(frame_indices, 0, total_frames - 1) + return frame_indices + +def get_audio_embed_bucket_fps(audio_embed, fps=16, batch_frames=81, m=0, video_rate=30): + num_layers, audio_frame_num, audio_dim = audio_embed.shape + return_all_layers = num_layers > 1 + scale = video_rate / fps + min_batch_num = int(audio_frame_num / (batch_frames * scale)) + 1 + bucket_num = min_batch_num * batch_frames + padd_audio_num = math.ceil(min_batch_num * batch_frames / fps * video_rate) - audio_frame_num + batch_idx = get_sample_indices(video_rate, audio_frame_num + padd_audio_num, fps, bucket_num, fixed_start=0) + batch_audio_eb = [] + audio_sample_stride = int(video_rate / fps) + for bi in batch_idx: + if bi < audio_frame_num: + chosen_idx = list(range(bi - m * audio_sample_stride, bi + (m + 1) * audio_sample_stride, audio_sample_stride)) + chosen_idx = [0 if c < 0 else c for c in chosen_idx] + chosen_idx = [audio_frame_num - 1 if c >= audio_frame_num else c for c in chosen_idx] + if return_all_layers: + frame_audio_embed = audio_embed[:, chosen_idx].flatten(start_dim=-2, end_dim=-1) + else: + frame_audio_embed = audio_embed[0][chosen_idx].flatten() + else: + frame_audio_embed = torch.zeros([audio_dim * (2 * m + 1)], device=audio_embed.device) if not return_all_layers else torch.zeros([num_layers, audio_dim * (2 * m + 1)], device=audio_embed.device) + batch_audio_eb.append(frame_audio_embed) + batch_audio_eb = torch.cat([c.unsqueeze(0) for c in batch_audio_eb], dim=0) + return batch_audio_eb, min_batch_num + +def xb_wan_sound_to_video(positive, negative, vae, width, height, length, batch_size, vae_tile_size, frame_offset=0, ref_image=None, audio_encoder_output=None, control_video=None, ref_motion=None, ref_motion_latent=None, scale_method="lanczos", crop_mode="center"): + latent_t = ((length - 1) // 4) + 1 + if audio_encoder_output is not None: + feat = torch.cat(audio_encoder_output["encoded_audio_all_layers"]) + video_rate = 30 + fps = 16 + feat = linear_interpolation(feat, input_fps=50, output_fps=video_rate) + batch_frames = latent_t * 4 + audio_embed_bucket, num_repeat = get_audio_embed_bucket_fps(feat, fps=fps, batch_frames=batch_frames, m=0, video_rate=video_rate) + audio_embed_bucket = audio_embed_bucket.unsqueeze(0) + if len(audio_embed_bucket.shape) == 3: + audio_embed_bucket = audio_embed_bucket.permute(0, 2, 1) + elif len(audio_embed_bucket.shape) == 4: + audio_embed_bucket = audio_embed_bucket.permute(0, 2, 3, 1) + + audio_embed_bucket = audio_embed_bucket[:, :, :, frame_offset:frame_offset + batch_frames] + if audio_embed_bucket.shape[3] > 0: + positive = node_helpers.conditioning_set_values(positive, {"audio_embed": audio_embed_bucket}) + negative = node_helpers.conditioning_set_values(negative, {"audio_embed": audio_embed_bucket * 0.0}) + frame_offset += batch_frames + + if ref_image is not None: + ref_image = comfy.utils.common_upscale(ref_image[:1].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + ref_latent = _encode_vae(vae, ref_image, vae_tile_size) + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) + + if ref_motion is not None: + if ref_motion.shape[0] > 73: + ref_motion = ref_motion[-73:] + ref_motion = comfy.utils.common_upscale(ref_motion.movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + if ref_motion.shape[0] < 73: + r = torch.ones([73, height, width, 3]) * 0.5 + r[-ref_motion.shape[0]:] = ref_motion + ref_motion = r + ref_motion_latent = _encode_vae(vae, ref_motion, vae_tile_size) + + if ref_motion is not None: + ref_motion_latent = ref_motion_latent[:, :, -19:] + positive = node_helpers.conditioning_set_values(positive, {"reference_motion": ref_motion_latent}) + negative = node_helpers.conditioning_set_values(negative, {"reference_motion": ref_motion_latent}) + + latent = torch.zeros([batch_size, 16, latent_t, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + + control_video_out = comfy.latent_formats.Wan21().process_out(torch.zeros_like(latent)) + if control_video is not None: + control_video = comfy.utils.common_upscale(control_video[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + control_video = _encode_vae(vae, control_video, vae_tile_size) + control_video_out[:, :, :control_video.shape[2]] = control_video + + positive = node_helpers.conditioning_set_values(positive, {"control_video": control_video_out}) + negative = node_helpers.conditioning_set_values(negative, {"control_video": control_video_out}) + + out_latent = {} + out_latent["samples"] = latent + return positive, negative, out_latent, frame_offset + +# ============================================================ +# XB_WanSoundImageToVideo — Wan 音频+图像转视频 +# ============================================================ +class XB_WanSoundImageToVideo: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 77, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), + "ref_image": ("IMAGE",), + "control_video": ("IMAGE",), + "ref_motion": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, ref_image=None, audio_encoder_output=None, control_video=None, ref_motion=None, scale_method="lanczos", crop_mode="center"): + positive, negative, out_latent, frame_offset = xb_wan_sound_to_video( + positive, negative, vae, width, height, length, batch_size, vae_tile_size, + ref_image=ref_image, audio_encoder_output=audio_encoder_output, + control_video=control_video, ref_motion=ref_motion, + scale_method=scale_method, crop_mode=crop_mode) + return (positive, negative, out_latent) + + +from comfy.ldm.wan.model_multitalk import InfiniteTalkOuterSampleWrapper, MultiTalkCrossAttnPatch, project_audio_features + +def _linear_interp(features, input_fps, output_fps, output_len=None): + features = features.transpose(1, 2) + seq_len = features.shape[2] / float(input_fps) + if output_len is None: + output_len = int(seq_len * output_fps) + output_features = torch.nn.functional.interpolate(features, size=output_len, align_corners=True, mode='linear') + return output_features.transpose(1, 2) + +def _process_infinite_talk_audio(model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=None, start_image=None, + audio_encoder_output_2=None, previous_frames=None, + mask_1=None, mask_2=None, segment_audio=False, + global_frame_offset=None, + scale_method="lanczos", crop_mode="center"): + """共用音频处理 + 模型补丁核心逻辑（单人/双人复用）""" + + # 🌟 零拷贝克隆：所有补丁打在 clone 上，OOM/取消中断时原模型不受污染 + model = model.clone() + + if previous_frames is not None and previous_frames.shape[0] < motion_frame_count: + raise ValueError("Not enough previous frames provided.") + + if audio_encoder_output_2 is not None: + if mask_1 is None or mask_2 is None: + raise ValueError("Masks must be provided if two audio encoder outputs are used.") + + ref_masks = None + if mask_1 is not None and mask_2 is not None: + if audio_encoder_output_2 is None: + raise ValueError("Second audio encoder output must be provided if two masks are used.") + ref_masks = torch.cat([mask_1, mask_2]) + + latent = torch.zeros([1, 16, ((length - 1) // 4) + 1, height // 8, width // 8], + device=comfy.model_management.intermediate_device()) + + concat_latent_image = None + if start_image is not None: + start_image = comfy.utils.common_upscale(start_image[:length].movedim(-1, 1), width, height, scale_method, crop_mode).movedim(1, -1) + image = torch.ones((length, height, width, start_image.shape[-1]), device=start_image.device, dtype=start_image.dtype) * 0.5 + image[:start_image.shape[0]] = start_image + + concat_latent_image = _encode_vae(vae, image, vae_tile_size) + concat_mask = torch.ones((1, 1, latent.shape[2] * 4, concat_latent_image.shape[-2], concat_latent_image.shape[-1]), + device=start_image.device, dtype=start_image.dtype) + concat_mask[:, :, :start_image.shape[0] + 3] = 0.0 + concat_mask = concat_mask.view(1, concat_mask.shape[2] // 4, 4, concat_mask.shape[3], concat_mask.shape[4]).transpose(1, 2) + + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + + # === 音频处理管线 === + encoded_audio_list = [] + seq_lengths = [] + for audio_encoder_output in [audio_encoder_output_1, audio_encoder_output_2]: + if audio_encoder_output is None: + continue + all_layers = audio_encoder_output["encoded_audio_all_layers"] + encoded_audio = torch.stack(all_layers, dim=0).squeeze(1)[1:] + encoded_audio = _linear_interp(encoded_audio, input_fps=50, output_fps=25).movedim(0, 1) + encoded_audio_list.append(encoded_audio) + seq_lengths.append(encoded_audio.shape[0]) + + multi_audio_type = "add" + if len(encoded_audio_list) > 1: + if multi_audio_type == "para": + max_len = max(seq_lengths) + padded = [] + for emb in encoded_audio_list: + if emb.shape[0] < max_len: + pad = torch.zeros(max_len - emb.shape[0], *emb.shape[1:], dtype=emb.dtype) + emb = torch.cat([emb, pad], dim=0) + padded.append(emb) + encoded_audio_list = padded + elif multi_audio_type == "add": + total_len = sum(seq_lengths) + full_list = [] + offset = 0 + for emb, seq_len in zip(encoded_audio_list, seq_lengths): + full = torch.zeros(total_len, *emb.shape[1:], dtype=emb.dtype) + full[offset:offset + seq_len] = emb + full_list.append(full) + offset += seq_len + encoded_audio_list = full_list + + token_ref_target_masks = None + if ref_masks is not None: + token_ref_target_masks = torch.nn.functional.interpolate( + ref_masks.unsqueeze(0), size=(latent.shape[-2] // 2, latent.shape[-1] // 2), mode='nearest')[0] + token_ref_target_masks = (token_ref_target_masks > 0).view(token_ref_target_masks.shape[0], -1) + + if previous_frames is not None: + motion_frames = comfy.utils.common_upscale(previous_frames[-motion_frame_count:].movedim(-1, 1), width, height, "bilinear", "center").movedim(1, -1) + # 🔧 优先使用显式传入的全局偏移量，彻底废弃通过 shape 反推 + if global_frame_offset is not None: + frame_offset = global_frame_offset + else: + frame_offset = previous_frames.shape[0] - motion_frame_count + if segment_audio: + audio_start = 0 # [fixed] + else: + audio_start = frame_offset + audio_end = audio_start + length + motion_frames_latent = _encode_vae(vae, motion_frames[:, :, :, :3], vae_tile_size) + trim_image = motion_frame_count + else: + # 🔧 即使没有 previous_frames（如硬切），也要尊重全局音频偏移量 + audio_start = global_frame_offset if global_frame_offset is not None else 0 + trim_image = 0 + audio_end = audio_start + length + motion_frames_latent = concat_latent_image[:, :, :1] if concat_latent_image is not None else torch.zeros((1, 16, 1, height // 8, width // 8), device=latent.device) + + audio_embed = project_audio_features(model_patch.model.audio_proj, encoded_audio_list, audio_start, audio_end).to(model.model_dtype()) + + + for key in ("infinite_talk_outer_sample",): + if hasattr(model, "wrappers") and key in model.wrappers: + del model.wrappers[key] + for key in ("attn2_patch",): + if hasattr(model, "object_patches") and key in model.object_patches: + del model.object_patches[key] + top = model.model_options.setdefault("transformer_options", {}) + top.pop("audio_embeds", None) + + top["audio_embeds"] = audio_embed + + model.add_wrapper_with_key( + comfy.patcher_extension.WrappersMP.OUTER_SAMPLE, + "infinite_talk_outer_sample", + InfiniteTalkOuterSampleWrapper(motion_frames_latent, model_patch, is_extend=previous_frames is not None)) + model.set_model_patch(MultiTalkCrossAttnPatch(model_patch, audio_scale), "attn2_patch") + + out_latent = {"samples": latent} + return model, positive, negative, out_latent, trim_image + + +# ============================================================ +# XB_WanInfiniteTalkToVideo_Single — Wan 单人语音转视频 +# ============================================================ +class XB_WanInfiniteTalkToVideo_Single: + """单人语音转视频分块 -- 对应 WanInfiniteTalkToVideo (single_speaker) + VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": ("MODEL",), + "model_patch": ("MODEL_PATCH",), + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), + "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "previous_frames": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") + RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=None, start_image=None, previous_frames=None, segment_audio=False, + global_frame_offset=None, + scale_method="lanczos", crop_mode="center"): + return _process_infinite_talk_audio( + model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=clip_vision_output, start_image=start_image, + previous_frames=previous_frames, segment_audio=segment_audio, + global_frame_offset=global_frame_offset) + + +# ============================================================ +# XB_WanInfiniteTalkToVideo_Dual — Wan 双人语音转视频 +# ============================================================ +class XB_WanInfiniteTalkToVideo_Dual: + """双人语音转视频分块 --对应 WanInfiniteTalkToVideo (two_speakers) + VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": ("MODEL",), + "model_patch": ("MODEL_PATCH",), + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), + "audio_encoder_output_2": ("AUDIO_ENCODER_OUTPUT",), + "mask_1": ("MASK",), + "mask_2": ("MASK",), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), + "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "previous_frames": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") + RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, audio_encoder_output_2, mask_1, mask_2, + motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=None, start_image=None, previous_frames=None, + global_frame_offset=None, + scale_method="lanczos", crop_mode="center"): + return _process_infinite_talk_audio( + model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=clip_vision_output, start_image=start_image, + audio_encoder_output_2=audio_encoder_output_2, + previous_frames=previous_frames, + mask_1=mask_1, mask_2=mask_2, + global_frame_offset=global_frame_offset, + scale_method=scale_method, crop_mode=crop_mode) + +# ============================================================================== +# 保留旧的 XB_WanInfiniteTalkToVideo 兼容别名（单人/双人 mode 选择器） +# ============================================================================== +# ============================================================ +# XB_WanInfiniteTalkToVideo — Wan 语音转视频分块 +# ============================================================ +class XB_WanInfiniteTalkToVideo: + """兼容旧工作流 --通过 mode 切换单人/双人""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "model": ("MODEL",), + "model_patch": ("MODEL_PATCH",), + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "mode": (["single_speaker", "two_speakers"], {"default": "single_speaker"}), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "audio_encoder_output_1": ("AUDIO_ENCODER_OUTPUT",), + "motion_frame_count": ("INT", {"default": 9, "min": 1, "max": 33, "step": 1}), + "audio_scale": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}), + "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "audio_encoder_output_2": ("AUDIO_ENCODER_OUTPUT",), + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "previous_frames": ("IMAGE",), + "mask_1": ("MASK",), + "mask_2": ("MASK",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("MODEL", "CONDITIONING", "CONDITIONING", "LATENT", "INT") + RETURN_NAMES = ("model", "positive", "negative", "latent", "trim_image") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, model, model_patch, positive, negative, vae, mode, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=None, start_image=None, + audio_encoder_output_2=None, previous_frames=None, + mask_1=None, mask_2=None, + global_frame_offset=None, + scale_method="lanczos", crop_mode="center"): + return _process_infinite_talk_audio( + model, model_patch, positive, negative, vae, width, height, length, + audio_encoder_output_1, motion_frame_count, audio_scale, vae_tile_size, + clip_vision_output=clip_vision_output, start_image=start_image, + audio_encoder_output_2=audio_encoder_output_2, + previous_frames=previous_frames, + mask_1=mask_1, mask_2=mask_2, + global_frame_offset=global_frame_offset, + scale_method=scale_method, crop_mode=crop_mode) + +# ============================================================================== + +# ============================================================================== + +# ============================================================================== + """Wan FunInpaint to Video with VAE tiling""" +# ============================================================================== +# ============================================================ +# XB_WanFunInpaintToVideo — Wan FunInpaint 转视频 +# ============================================================ +class XB_WanFunInpaintToVideo: + """Wan FunInpaint to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "end_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, + start_image=None, end_image=None, clip_vision_output=None, + scale_method="lanczos", crop_mode="center"): + flfv = XB_WanFirstLastFrameToVideo() + return flfv.process(positive, negative, vae, width, height, length, batch_size, vae_tile_size, + start_image=start_image, end_image=end_image, + clip_vision_start_image=clip_vision_output, + scale_method=scale_method, crop_mode=crop_mode) + +# ============================================================================== + """Wan Camera Control to Video with VAE tiling""" +# ============================================================================== +# ============================================================ +# XB_WanCameraImageToVideo — Wan 相机运镜图生视频 +# ============================================================ +class XB_WanCameraImageToVideo: + """Wan Camera Control to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "camera_conditions": ("WAN_CAMERA_EMBEDDING",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, + start_image=None, clip_vision_output=None, camera_conditions=None, + scale_method="lanczos", crop_mode="center"): + latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + concat_latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + concat_latent = comfy.latent_formats.Wan21().process_out(concat_latent) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + concat_latent_image = _encode_vae(vae, start_image[:, :, :, :3], vae_tile_size) + concat_latent[:,:,:concat_latent_image.shape[2]] = concat_latent_image[:,:,:concat_latent.shape[2]] + mask = torch.ones((1, 1, latent.shape[2] * 4, latent.shape[-2], latent.shape[-1]), device=latent.device) + mask[:, :, :start_image.shape[0] + 3] = 0.0 + mask = mask.view(1, mask.shape[2] // 4, 4, mask.shape[3], mask.shape[4]).transpose(1, 2) + positive = node_helpers.conditioning_set_values(positive, {"concat_latent_image": concat_latent, "concat_mask": mask}) + negative = node_helpers.conditioning_set_values(negative, {"concat_latent_image": concat_latent, "concat_mask": mask}) + + if camera_conditions is not None: + positive = node_helpers.conditioning_set_values(positive, {'camera_conditions': camera_conditions}) + negative = node_helpers.conditioning_set_values(negative, {'camera_conditions': camera_conditions}) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + + out_latent = {"samples": latent} + return (positive, negative, out_latent) + +# ============================================================================== + """Wan Phantom Subject to Video with VAE tiling""" +# ============================================================================== +# ============================================================ +# XB_WanPhantomSubjectToVideo — Wan Phantom 主体转视频 +# ============================================================ +class XB_WanPhantomSubjectToVideo: + """Wan Phantom Subject to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "images": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative_text", "negative_img_text", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, + images=None, scale_method="lanczos", crop_mode="center"): + latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + cond2 = negative + if images is not None: + images = _upscale(images[:length], width, height, scale_method, crop_mode) + latent_images = [] + for i in images: + latent_images += [_encode_vae(vae, i.unsqueeze(0)[:, :, :, :3], vae_tile_size)] + concat_latent_image = torch.cat(latent_images, dim=2) + positive = node_helpers.conditioning_set_values(positive, {"time_dim_concat": concat_latent_image}) + cond2 = node_helpers.conditioning_set_values(negative, {"time_dim_concat": concat_latent_image}) + negative = node_helpers.conditioning_set_values(negative, {"time_dim_concat": comfy.latent_formats.Wan21().process_out(torch.zeros_like(concat_latent_image))}) + + out_latent = {"samples": latent} + return (positive, cond2, negative, out_latent) + +# ============================================================================== + """Wan HuMo to Video with VAE tiling""" +# ============================================================================== +def _get_audio_emb_window(audio_emb, frame_num, frame0_idx, audio_shift=2): + zero_audio_embed = torch.zeros((audio_emb.shape[1], audio_emb.shape[2]), dtype=audio_emb.dtype, device=audio_emb.device) + zero_audio_embed_3 = torch.zeros((3, audio_emb.shape[1], audio_emb.shape[2]), dtype=audio_emb.dtype, device=audio_emb.device) + iter_ = 1 + (frame_num - 1) // 4 + audio_emb_wind = [] + for lt_i in range(iter_): + if lt_i == 0: + st = frame0_idx + lt_i - 2 + ed = frame0_idx + lt_i + 3 + wind_feat = torch.stack([audio_emb[i] if (0 <= i < audio_emb.shape[0]) else zero_audio_embed for i in range(st, ed)], dim=0) + wind_feat = torch.cat((zero_audio_embed_3, wind_feat), dim=0) + else: + st = frame0_idx + 1 + 4 * (lt_i - 1) - audio_shift + ed = frame0_idx + 1 + 4 * lt_i + audio_shift + wind_feat = torch.stack([audio_emb[i] if (0 <= i < audio_emb.shape[0]) else zero_audio_embed for i in range(st, ed)], dim=0) + audio_emb_wind.append(wind_feat) + audio_emb_wind = torch.stack(audio_emb_wind, dim=0) + return audio_emb_wind, ed - audio_shift + +# ============================================================ +# XB_WanHuMoImageToVideo — Wan HuMo 图生视频 +# ============================================================ +class XB_WanHuMoImageToVideo: + """Wan HuMo to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 97, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), + "ref_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, vae_tile_size, + ref_image=None, audio_encoder_output=None, scale_method="lanczos", crop_mode="center"): + latent_t = ((length - 1) // 4) + 1 + latent = torch.zeros([batch_size, 16, latent_t, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + + if ref_image is not None: + ref_image = _upscale(ref_image[:1], width, height, scale_method, crop_mode) + ref_latent = _encode_vae(vae, ref_image[:, :, :, :3], vae_tile_size) + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [torch.zeros_like(ref_latent)]}, append=True) + else: + zero_latent = torch.zeros([batch_size, 16, 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [zero_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [zero_latent]}, append=True) + + if audio_encoder_output is not None: + audio_emb = torch.stack(audio_encoder_output["encoded_audio_all_layers"], dim=2) + audio_len = audio_encoder_output["audio_samples"] // 640 + audio_emb = audio_emb[:, :audio_len * 2] + feat0 = linear_interpolation(audio_emb[:, :, 0: 8].mean(dim=2), 50, 25) + feat1 = linear_interpolation(audio_emb[:, :, 8: 16].mean(dim=2), 50, 25) + feat2 = linear_interpolation(audio_emb[:, :, 16: 24].mean(dim=2), 50, 25) + feat3 = linear_interpolation(audio_emb[:, :, 24: 32].mean(dim=2), 50, 25) + feat4 = linear_interpolation(audio_emb[:, :, 32], 50, 25) + audio_emb = torch.stack([feat0, feat1, feat2, feat3, feat4], dim=2)[0] + audio_emb, _ = _get_audio_emb_window(audio_emb, length, frame0_idx=0) + audio_emb = audio_emb.unsqueeze(0) + audio_emb_neg = torch.zeros_like(audio_emb) + positive = node_helpers.conditioning_set_values(positive, {"audio_embed": audio_emb}) + negative = node_helpers.conditioning_set_values(negative, {"audio_embed": audio_emb_neg}) + else: + zero_audio = torch.zeros([batch_size, latent_t + 1, 8, 5, 1280], device=comfy.model_management.intermediate_device()) + positive = node_helpers.conditioning_set_values(positive, {"audio_embed": zero_audio}) + negative = node_helpers.conditioning_set_values(negative, {"audio_embed": zero_audio}) + + out_latent = {"samples": latent} + return (positive, negative, out_latent) + +# ============================================================================== +# 7.5.5 🎬 Wan22 ImageToVideo Latent 分块 +# ============================================================================== +# ============================================================ +# XB_Wan22ImageToVideoLatent — Wan 2.2 图生视频 Latent +# ============================================================ +class XB_Wan22ImageToVideoLatent: + """Wan 2.2 Image to Video Latent with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "vae": ("VAE",), + "width": ("INT", {"default": 1280, "min": 16, "max": 8192, "step": 32}), + "height": ("INT", {"default": 704, "min": 32, "max": 8192, "step": 32}), + "length": ("INT", {"default": 49, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "start_image": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("LATENT",) + RETURN_NAMES = ("latent",) + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, vae, width, height, length, batch_size, vae_tile_size, + start_image=None, scale_method="lanczos", crop_mode="center"): + latent = torch.zeros([1, 48, ((length - 1) // 4) + 1, height // 16, width // 16], device=comfy.model_management.intermediate_device()) + mask = torch.ones([latent.shape[0], 1, ((length - 1) // 4) + 1, latent.shape[-2], latent.shape[-1]], device=comfy.model_management.intermediate_device()) + + if start_image is not None: + start_image = _upscale(start_image[:length], width, height, scale_method, crop_mode) + latent_temp = _encode_vae(vae, start_image, vae_tile_size) + latent[:, :, :latent_temp.shape[-3]] = latent_temp + mask[:, :, :latent_temp.shape[-3]] *= 0.0 + + out_latent = {} + latent_format = comfy.latent_formats.Wan22() + latent = latent_format.process_out(latent) * mask + latent * (1.0 - mask) + out_latent["samples"] = latent.repeat((batch_size,) + (1,) * (latent.ndim - 1)) + out_latent["noise_mask"] = mask.repeat((batch_size,) + (1,) * (mask.ndim - 1)) + return (out_latent,) + +# ============================================================================== +# 7.5.6 🎬 Wan SoundImage Extend 分块 +# ============================================================================== +# ============================================================ +# XB_WanSoundImageToVideoExtend — Wan 音频+图像扩展视频 +# ============================================================ +class XB_WanSoundImageToVideoExtend: + """Wan Sound Extend to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "length": ("INT", {"default": 77, "min": 1, "max": 8192, "step": 4}), + "video_latent": ("LATENT",), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), + "ref_image": ("IMAGE",), + "control_video": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, length, video_latent, vae_tile_size, + ref_image=None, audio_encoder_output=None, control_video=None, + scale_method="lanczos", crop_mode="center"): + video_latent_data = video_latent["samples"] + width = video_latent_data.shape[-1] * 8 + height = video_latent_data.shape[-2] * 8 + batch_size = video_latent_data.shape[0] + frame_offset = video_latent_data.shape[-3] * 4 + positive, negative, out_latent, frame_offset = xb_wan_sound_to_video( + positive, negative, vae, width, height, length, batch_size, vae_tile_size, + frame_offset=frame_offset, ref_image=ref_image, + audio_encoder_output=audio_encoder_output, + control_video=control_video, ref_motion=None, ref_motion_latent=video_latent_data, + scale_method=scale_method, crop_mode=crop_mode) + return (positive, negative, out_latent) + +# ============================================================================== + """Wan SCAIL Pose to Video with VAE tiling""" +# ============================================================================== +# ============================================================ +# XB_WanSCAILToVideo — Wan SCAIL 转视频 +# ============================================================ +class XB_WanSCAILToVideo: + """Wan SCAIL Pose to Video with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), + "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "reference_image": ("IMAGE",), + "pose_video": ("IMAGE",), + "scale_method": (_SCALE_METHODS, {"default": "lanczos"}), + "crop_mode": (_CROP_MODES, {"default": "center"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, pose_strength, pose_start, pose_end, vae_tile_size, + reference_image=None, clip_vision_output=None, pose_video=None, + scale_method="lanczos", crop_mode="center"): + latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + + if reference_image is not None: + reference_image = _upscale(reference_image[:1], width, height, scale_method, crop_mode) + ref_latent = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [torch.zeros_like(ref_latent)]}, append=True) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + + if pose_video is not None: + pose_video = _upscale(pose_video[:length], width // 2, height // 2, "area", "center") + pose_video_latent = _encode_vae(vae, pose_video[:, :, :, :3], vae_tile_size) * pose_strength + positive = node_helpers.conditioning_set_values_with_timestep_range(positive, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) + negative = node_helpers.conditioning_set_values_with_timestep_range(negative, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) + + out_latent = {"samples": latent} + return (positive, negative, out_latent) + +# ============================================================================== +# 7.5.7 Pro 🎬 Wan SCAIL (pose) 转视频分块 — 高配版 +# ============================================================================== +# SCAIL-2 mask helper — colored RGB mask → 28-channel binary latent +# ============================================================================== +def _scail_extract_mask_to_28ch(rgb_video): + """Colored RGB mask (T, H, W, 3) in [0,1] -> SCAIL-2 28-channel binary latent + (1, T_lat, 28, H_lat, W_lat). 7 per-color binary channels (white/r/g/b/y/m/c) + threshold-extracted at 225/255, 8x spatial downsample, 4-frame temporal stacking.""" + T, H, W, _ = rgb_video.shape + _ON_THRESH = 225.0 / 255.0 + mask = rgb_video.movedim(-1, 1).float() + R = (mask[:, 0:1] > _ON_THRESH).float() + G = (mask[:, 1:2] > _ON_THRESH).float() + B = (mask[:, 2:3] > _ON_THRESH).float() + nR, nG, nB = 1 - R, 1 - G, 1 - B + binary_7ch = torch.cat([ + R * G * B, # white + R * nG * nB, # red + nR * G * nB, # green + nR * nG * B, # blue + R * G * nB, # yellow + R * nG * B, # magenta + nR * G * B, # cyan + ], dim=1) + H_lat, W_lat = H, W + for _ in range(3): + H_lat = (H_lat + 1) // 2 + W_lat = (W_lat + 1) // 2 + binary_7ch = torch.nn.functional.interpolate(binary_7ch, size=(H_lat, W_lat), mode='area') + T_latent = (T - 1) // 4 + 1 + padded = torch.cat([binary_7ch[:1].repeat(4, 1, 1, 1), binary_7ch[1:]], dim=0) + out = padded.view(T_latent, 28, H_lat, W_lat) + return out.unsqueeze(0) + +# ============================================================================== +# 7.5.7 Pro 🎬 Wan SCAIL-2 (pose) 转视频分块 — 完整版 +# ============================================================================== +# ============================================================ +# XB_WanSCAILToVideoPro — Wan SCAIL Pro 转视频 +# ============================================================ +class XB_WanSCAILToVideoPro: + """Wan SCAIL-2 Pose to Video Pro — exact port from ComfyUI core nodes_scail.py with VAE tiling""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 512, "min": 16, "max": 8192, "step": 32}), + "height": ("INT", {"default": 896, "min": 32, "max": 8192, "step": 32}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + "pose_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}), + "pose_start": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "pose_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}), + "replacement_mode": ("BOOLEAN", {"default": False}), + "video_frame_offset": ("INT", {"default": 0, "min": 0, "max": 8192, "step": 1}), + "previous_frame_count": ("INT", {"default": 5, "min": 1, "max": 8192, "step": 4}), + "vae_tile_size": ("INT", {"default": 64, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "pose_video": ("IMAGE",), + "pose_video_mask": ("IMAGE",), + "reference_image": ("IMAGE",), + "reference_image_mask": ("IMAGE",), + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "previous_frames": ("IMAGE",), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT", "INT") + RETURN_NAMES = ("positive", "negative", "latent", "video_frame_offset") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, batch_size, pose_strength, pose_start, pose_end, + replacement_mode, video_frame_offset, previous_frame_count, vae_tile_size, + pose_video=None, pose_video_mask=None, reference_image=None, + reference_image_mask=None, clip_vision_output=None, previous_frames=None): + latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], device=comfy.model_management.intermediate_device()) + noise_mask = None + + # ref_mask_flag: False=Animation Mode, True=Replacement Mode + ref_mask_flag = not replacement_mode + positive = node_helpers.conditioning_set_values(positive, {"ref_mask_flag": ref_mask_flag}) + negative = node_helpers.conditioning_set_values(negative, {"ref_mask_flag": ref_mask_flag}) + + # --- previous_frames --- + prev_trimmed = None + if previous_frames is not None and previous_frames.shape[0] > 0: + prev_trimmed = previous_frames[-previous_frame_count:] + video_frame_offset -= prev_trimmed.shape[0] + video_frame_offset = max(0, video_frame_offset) + + # --- reference_image + mask --- + if reference_image is not None: + reference_image = comfy.utils.common_upscale(reference_image[:1].movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) + # Replacement Mode: composite ref on black bg using reference_image_mask as alpha matte + if replacement_mode and reference_image_mask is not None: + rm = comfy.utils.common_upscale(reference_image_mask[:1].movedim(-1, 1), width, height, "nearest-exact", "center").movedim(1, -1) + is_char = (rm[..., :3].max(dim=-1, keepdim=True).values > 0.1).to(reference_image.dtype) + reference_image = reference_image * is_char + ref_latent = _encode_vae(vae, reference_image[:, :, :, :3], vae_tile_size) + positive = node_helpers.conditioning_set_values(positive, {"reference_latents": [ref_latent]}, append=True) + negative = node_helpers.conditioning_set_values(negative, {"reference_latents": [ref_latent]}, append=True) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values(positive, {"clip_vision_output": clip_vision_output}) + negative = node_helpers.conditioning_set_values(negative, {"clip_vision_output": clip_vision_output}) + + # --- pose_video (with offset truncation) --- + if pose_video is not None: + if pose_video.shape[0] <= video_frame_offset: + pose_video = None + else: + pose_video = pose_video[video_frame_offset:] + if pose_video_mask is not None: + if pose_video_mask.shape[0] <= video_frame_offset: + pose_video_mask = None + else: + pose_video_mask = pose_video_mask[video_frame_offset:] + + # Truncate pose+mask jointly to the shorter of the two, capped at length + ts = [v.shape[0] for v in (pose_video, pose_video_mask) if v is not None] + if ts: + T_kept = ((min(min(ts), length) - 1) // 4) * 4 + 1 + if pose_video is not None: + pose_video = pose_video[:T_kept] + if pose_video_mask is not None: + pose_video_mask = pose_video_mask[:T_kept] + + if pose_video is not None: + pose_video = comfy.utils.common_upscale(pose_video[:length].movedim(-1, 1), width // 2, height // 2, "area", "center").movedim(1, -1) + pose_video_latent = _encode_vae(vae, pose_video[:, :, :, :3], vae_tile_size) * pose_strength + positive = node_helpers.conditioning_set_values_with_timestep_range(positive, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) + negative = node_helpers.conditioning_set_values_with_timestep_range(negative, {"pose_video_latent": pose_video_latent}, pose_start, pose_end) + + if pose_video_mask is not None: + mask_video_hw = comfy.utils.common_upscale(pose_video_mask[:length].movedim(-1, 1), width // 2, height // 2, "area", "center").movedim(1, -1) + driving_mask_28ch = _scail_extract_mask_to_28ch(mask_video_hw) + positive = node_helpers.conditioning_set_values(positive, {"driving_mask_28ch": driving_mask_28ch}) + negative = node_helpers.conditioning_set_values(negative, {"driving_mask_28ch": driving_mask_28ch}) + + if reference_image_mask is not None and replacement_mode: + ref_mask_hw = comfy.utils.common_upscale(reference_image_mask[:1].movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) + ref_mask_1f = _scail_extract_mask_to_28ch(ref_mask_hw) + zeros = torch.zeros((1, latent.shape[2], 28, ref_mask_1f.shape[-2], ref_mask_1f.shape[-1]), device=ref_mask_1f.device, dtype=ref_mask_1f.dtype) + ref_mask_28ch = torch.cat([ref_mask_1f, zeros], dim=1) + positive = node_helpers.conditioning_set_values(positive, {"ref_mask_28ch": ref_mask_28ch}) + negative = node_helpers.conditioning_set_values(negative, {"ref_mask_28ch": ref_mask_28ch}) + + if prev_trimmed is not None: + pf = comfy.utils.common_upscale(prev_trimmed.movedim(-1, 1), width, height, "bicubic", "center").movedim(1, -1) + prev_latent = _encode_vae(vae, pf[:, :, :, :3], vae_tile_size) + prev_latent_frames = min(prev_latent.shape[2], latent.shape[2]) + latent[:, :, :prev_latent_frames] = prev_latent[:, :, :prev_latent_frames].to(latent.dtype) + noise_mask = torch.ones((1, 1, latent.shape[2], latent.shape[-2], latent.shape[-1]), device=latent.device, dtype=latent.dtype) + noise_mask[:, :, :prev_latent_frames] = 0.0 + + out_latent = {"samples": latent} + if noise_mask is not None: + out_latent["noise_mask"] = noise_mask + return (positive, negative, out_latent, video_frame_offset + length) + +# ============================================================================== +# 7.6 🎬 WanDancerVideo — Wan Dancer 音视频生成 (VAE 分块版) +# ============================================================================== +# ============================================================ +# XB_WanDancerVideo — Wan Dancer 转视频 (VAE 分块) +# ============================================================ +class XB_WanDancerVideo: + """Wan Dancer Video with VAE tiling —— 完全复刻官方 WanDancerVideo 逻辑， + 仅将 vae.encode() 替换为 _encode_vae() 以支持空间/时间分块。""" + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 149, "min": 1, "max": 8192, "step": 4, "tooltip": "生成视频的总帧数"}), + "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + }, + "optional": { + "clip_vision_output": ("CLIP_VISION_OUTPUT",), + "clip_vision_output_ref": ("CLIP_VISION_OUTPUT",), + "start_image": ("IMAGE",), + "mask": ("MASK",), + "audio_encoder_output": ("AUDIO_ENCODER_OUTPUT",), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Pipeline" + + def process(self, positive, negative, vae, width, height, length, vae_tile_size, + start_image=None, mask=None, + clip_vision_output=None, clip_vision_output_ref=None, + audio_encoder_output=None): + latent = torch.zeros([1, 16, ((length - 1) // 4) + 1, height // 8, width // 8], + device=comfy.model_management.intermediate_device()) + + if start_image is not None: + start_image = comfy.utils.common_upscale( + start_image[:length].movedim(-1, 1), width, height, "bilinear", "center" + ).movedim(1, -1) + image = torch.zeros((length, height, width, start_image.shape[-1]), + device=start_image.device, dtype=start_image.dtype) + image[:start_image.shape[0]] = start_image + + # 🔧 核心改动：使用 _encode_vae 替代 vae.encode()，支持空间/时间分块 + concat_latent_image = _encode_vae(vae, image[:, :, :, :3], vae_tile_size) + + if mask is None: + concat_mask = torch.ones( + (1, 1, latent.shape[2], concat_latent_image.shape[-2], concat_latent_image.shape[-1]), + device=start_image.device, dtype=start_image.dtype) + concat_mask[:, :, :((start_image.shape[0] - 1) // 4) + 1] = 0.0 + else: + concat_mask = 1 - mask[:length].unsqueeze(0) + concat_mask = comfy.utils.common_upscale( + concat_mask, concat_latent_image.shape[-2], concat_latent_image.shape[-1], + "nearest-exact", "disabled") + concat_mask = torch.cat( + [torch.repeat_interleave(concat_mask[:, 0:1], repeats=4, dim=1), concat_mask[:, 1:]], dim=1) + concat_mask = concat_mask.view( + 1, concat_mask.shape[1] // 4, 4, + concat_latent_image.shape[-2], concat_latent_image.shape[-1] + ).transpose(1, 2) + + positive = node_helpers.conditioning_set_values( + positive, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) + negative = node_helpers.conditioning_set_values( + negative, {"concat_latent_image": concat_latent_image, "concat_mask": concat_mask}) + + if clip_vision_output is not None: + positive = node_helpers.conditioning_set_values( + positive, {"clip_vision_output": clip_vision_output, "clip_vision_output_ref": clip_vision_output_ref}) + negative = node_helpers.conditioning_set_values( + negative, {"clip_vision_output": clip_vision_output, "clip_vision_output_ref": clip_vision_output_ref}) + + if audio_encoder_output is not None: + positive = node_helpers.conditioning_set_values(positive, { + "audio_embed": audio_encoder_output["audio_feature"], + "fps": audio_encoder_output["fps"], + "audio_inject_scale": audio_encoder_output.get("audio_inject_scale", 1.0) + }) + negative = node_helpers.conditioning_set_values(negative, { + "audio_embed": audio_encoder_output["audio_feature"], + "fps": audio_encoder_output["fps"], + "audio_inject_scale": audio_encoder_output.get("audio_inject_scale", 1.0) + }) + + out_latent = {"samples": latent} + return (positive, negative, out_latent) + +# ============================================================ +# XB_WanDanceSwitcher — Wan 舞蹈切换器 +# ============================================================ +class XB_WanDanceSwitcher: + """Wan 舞蹈切换器 — 选择舞蹈类型和动作幅度，输出对应提示词。 + + 替换原工作流中 CustomCombo + RegexExtract + StringReplace + StringConcatenate 节点组， + 简化为一键切换。输出与原版完全一致的中文提示词。 + """ + _DANCE_STYLES = [ + "古典舞", + "韩舞", + "街舞", + "拉丁舞", + "踢踏舞", + ] + _AMPLITUDES = [ + "低", + "中等", + "高", + "最大", + ] + + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "dance_style": (cls._DANCE_STYLES, {"default": "古典舞"}), + "motion_amplitude": (cls._AMPLITUDES, {"default": "中等"}), + } + } + + RETURN_TYPES = ("STRING", "STRING") + RETURN_NAMES = ("舞蹈风格提示词", "整体描述提示词") + FUNCTION = "switch" + CATEGORY = "XB_ToolBox/Wan" + + def switch(self, dance_style, motion_amplitude): + # 输出1 (接18号节点): 舞蹈风格名称（原版 RegexExtract 输出） + # 输出2 (接20号节点): 整体描述提示词（原版 StringConcatenate 输出） + dance_prompt = f"一个人正在跳舞，舞蹈种类是{dance_style}" + full_prompt = f"{dance_prompt},图像清晰程度高,人物动作幅度{motion_amplitude}" + + return (dance_style, full_prompt) + +# ============================================================ +# XB_WanDancerCombo — Wan Dancer 综合输入 (音频切片 + 特征提取 + 舞蹈选择) +# ============================================================ +import os as _os +import folder_paths as _folder_paths + +class XB_WanDancerCombo: + """Wan Dancer 综合输入节点 —— 融合音频切片 + 音频特征提取 + 舞蹈切换器。 + + 界面照抄 AudioSlicerV1，内部集成了 WanDancerEncodeAudio 的全套音频特征提取 + 和 XB_WanDanceSwitcher 的舞蹈类型/动作幅度选择。 + + 输入: 音频文件 + 起止时间 + 视频帧数 + 注入比例 + 舞蹈类型 + 动作幅度 + 输出: 切片音频 + 舞蹈风格提示词(含FPS) + 整体描述提示词 + """ + _DANCE_STYLES = ["古典舞", "韩舞", "街舞", "拉丁舞", "踢踏舞"] + _AMPLITUDES = ["低", "中等", "高", "最大"] + + @classmethod + def INPUT_TYPES(cls): + input_dir = _folder_paths.get_input_directory() + audio_files = ["none"] + if _os.path.exists(input_dir): + for f in sorted(_os.listdir(input_dir)): + if f.lower().endswith(('.mp3','.wav','.flac','.ogg','.m4a','.aac','.opus','.wma','.webm')): + audio_files.append(f) + + return { + "required": { + "audio": (audio_files,), + "start_time": ("FLOAT", {"default": 0.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "end_time": ("FLOAT", {"default": 10.00, "min": 0.00, "max": 99999.00, "step": 0.01}), + "duration_display": ("STRING", {"default": "0 帧", "multiline": False}), + "video_frames": ("INT", {"default": 149, "min": 1, "max": 8192, "step": 4, "tooltip": "视频总帧数"}), + "audio_inject_scale": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01, "tooltip": "音频注入强度"}), + "dance_style": (cls._DANCE_STYLES, {"default": "古典舞"}), + "motion_amplitude": (cls._AMPLITUDES, {"default": "中等"}), + } + } + + RETURN_TYPES = ("AUDIO", "STRING", "STRING") + RETURN_NAMES = ("音频", "整体描述提示词", "舞蹈风格提示词") + FUNCTION = "process" + CATEGORY = "XB_ToolBox/Wan" + + @classmethod + def VALIDATE_INPUTS(cls, audio, **kwargs): + return True + + def process(self, audio, start_time, end_time, duration_display, + video_frames, audio_inject_scale, dance_style, motion_amplitude): + # ── 1. 音频切片 (照抄 AudioSlicerV1) ── + from ..audio.nodes_audio_slicer import _load_audio_file, _make_audio, _snap_4n1 + + if audio == "none": + waveform = torch.zeros((1, 1), dtype=torch.float32) + sample_rate = 44100 + else: + result = _load_audio_file(_os.path.join(_folder_paths.get_input_directory(), audio)) + if result is None: + waveform = torch.zeros((1, 1), dtype=torch.float32) + sample_rate = 44100 + else: + waveform, sample_rate = result + + if waveform.shape[0] > 1: + waveform = waveform.mean(dim=0, keepdim=True) + + total_duration = waveform.shape[1] / sample_rate + start_time = max(0.0, min(start_time, total_duration)) + end_time = max(start_time, min(end_time, total_duration)) + + start_sample = int(start_time * sample_rate) + end_sample = int(end_time * sample_rate) + if end_sample <= start_sample: + end_sample = min(start_sample + int(0.1 * sample_rate), waveform.shape[1]) + + sliced_waveform = waveform[:, start_sample:end_sample] + + # ── 2. 音频特征提取 (照抄 WanDancerEncodeAudio) ── + try: + import torchaudio as _torchaudio + except ImportError: + raise ImportError( + "\n❌ XB_WanDancerCombo 需要 torchaudio 库。\n" + " 请运行: pip install torchaudio" + ) + try: + from comfy_extras.nodes_wandancer import ( + quick_tempo_estimate, estimate_tempo_from_onset, + _compute_mel_spectrogram, power_to_db, compute_onset_envelope, + compute_mfcc, compute_chroma_cens, detect_onset_peaks, + track_beats + ) + except ImportError as e: + raise ImportError( + "\n❌ XB_WanDancerCombo 需要 ComfyUI 内置的 WanDancer 模块\n" + " (comfy_extras.nodes_wandancer)，但当前版本未找到。\n" + " 请升级 ComfyUI 到 0.28.0 或更新版本！\n" + f" 原始错误: {e}" + ) + + try: + import scipy.signal, scipy.ndimage, scipy.fft, scipy.sparse # noqa: F401 WanDancer 特征提取依赖 + except ImportError: + raise ImportError( + "\n❌ XB_WanDancerCombo 需要 scipy 库。\n" + " 请运行: pip install scipy" + ) + + wf_enc = sliced_waveform.squeeze() + sr_enc = sample_rate + base_fps = 30 + hop_length = 512 + model_sr = 22050 + n_fft = 2048 + + start_bpm = quick_tempo_estimate(wf_enc.cpu().numpy(), sr_enc, hop_length=hop_length) + + resample_sr = base_fps * hop_length + wf_enc = _torchaudio.functional.resample(wf_enc.unsqueeze(0), sr_enc, resample_sr).squeeze(0) + + waveform_np = wf_enc.cpu().numpy() + mel_spec = _compute_mel_spectrogram(waveform_np, model_sr, n_fft, hop_length, n_mels=128) + mel_spec_db = power_to_db(mel_spec, amin=1e-10, top_db=80.0, ref=1.0) + envelope = compute_onset_envelope(mel_spec_db, n_fft, hop_length) + mfcc = compute_mfcc(mel_spec_db, n_mfcc=20) + chroma = compute_chroma_cens(y=waveform_np, sr=model_sr, hop_length=hop_length).T + peak_idxs = detect_onset_peaks(envelope, sr=model_sr, hop_length=hop_length) + peak_onehot = np.zeros_like(envelope, dtype=np.float32) + peak_onehot[peak_idxs] = 1.0 + beat_tracking_tempo = estimate_tempo_from_onset(envelope, sr=model_sr, hop_length=hop_length, start_bpm=start_bpm) + beat_idxs = track_beats(envelope, beat_tracking_tempo, model_sr, hop_length, tightness=100, trim=True) + beat_onehot = np.zeros_like(envelope, dtype=np.float32) + beat_onehot[beat_idxs] = 1.0 + + audio_feature = np.concatenate( + [envelope[:, None], mfcc, chroma, peak_onehot[:, None], beat_onehot[:, None]], axis=-1) + audio_feature = torch.from_numpy(audio_feature).unsqueeze(0).to(comfy.model_management.intermediate_device()) + + # fps 基于切片后音频的特征长度计算（与原版 WanDancerEncodeAudio 逻辑完全一致） + fps = float(base_fps / int(audio_feature.shape[1] / video_frames + 0.5)) + if int(fps + 0.5) != 30: + fps_string = " 帧率是{:.4f}".format(fps) + else: + fps_string = ", 帧率是30fps。" + + # ── 3. 构建音频输出 (含 audio_encoder_output 特征) ── + audio_out = { + "waveform": sliced_waveform.unsqueeze(0).unsqueeze(0), + "sample_rate": sample_rate, + "audio_feature": audio_feature, + "fps": fps, + "audio_inject_scale": audio_inject_scale, + } + + # ── 4. 提示词生成 (照抄 WanDanceSwitcher) ── + dance_prompt = f"{dance_style}{fps_string}" + full_prompt = f"一个人正在跳舞，舞蹈种类是{dance_style},图像清晰程度高,人物动作幅度{motion_amplitude}" + + return (audio_out, full_prompt, dance_prompt) + +# ============================================================ +# XB_BerniniConditioning - Bernini VAE分块编码 +# ============================================================ + +def _resize_long_edge(image, max_size, stride=16, method="area"): + h, w = image.shape[1], image.shape[2] + scale = min(max_size / max(h, w), 1.0) + nh = max(stride, round(h * scale / stride) * stride) + nw = max(stride, round(w * scale / stride) * stride) + return comfy.utils.common_upscale(image[:, :, :, :3].movedim(-1, 1), nw, nh, method, "disabled").movedim(1, -1) + + +class XB_BerniniConditioning: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "positive": ("CONDITIONING",), + "negative": ("CONDITIONING",), + "vae": ("VAE",), + "width": ("INT", {"default": 832, "min": 16, "max": 8192, "step": 16}), + "height": ("INT", {"default": 480, "min": 16, "max": 8192, "step": 16}), + "length": ("INT", {"default": 81, "min": 1, "max": 8192, "step": 4}), + "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096}), + }, + "optional": { + "source_video": ("IMAGE",), + "reference_video": ("IMAGE",), + "reference_image_0": ("IMAGE",), + "reference_image_1": ("IMAGE",), + "reference_image_2": ("IMAGE",), + "reference_image_3": ("IMAGE",), + "reference_image_4": ("IMAGE",), + "reference_image_5": ("IMAGE",), + "reference_image_6": ("IMAGE",), + "reference_image_7": ("IMAGE",), + "ref_max_size": ("INT", {"default": 848, "min": 16, "max": 8192, "step": 16}), + "vae_tile_size": ("INT", {"default": 256, "min": 64, "max": 3840, "step": 32}), + "scale_method": (_SCALE_METHODS, {"default": "area"}), + } + } + + RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT") + RETURN_NAMES = ("positive", "negative", "latent") + FUNCTION = "go" + CATEGORY = "XB_ToolBox/Wan" + + def go(self, positive, negative, vae, width, height, length, batch_size, + source_video=None, reference_video=None, + reference_image_0=None, reference_image_1=None, + reference_image_2=None, reference_image_3=None, + reference_image_4=None, reference_image_5=None, + reference_image_6=None, reference_image_7=None, + ref_max_size=848, vae_tile_size=256, scale_method="area"): + latent = torch.zeros([batch_size, 16, ((length - 1) // 4) + 1, height // 8, width // 8], + device=comfy.model_management.intermediate_device()) + context = [] + if source_video is not None: + vid = comfy.utils.common_upscale(source_video[:length, :, :, :3].movedim(-1, 1), width, height, scale_method, "center").movedim(1, -1) + context.append(_encode_vae(vae, vid[:, :, :, :3], vae_tile_size)) + if reference_video is not None: + ref_vid = _resize_long_edge(reference_video[:length], ref_max_size, method=scale_method) + context.append(_encode_vae(vae, ref_vid[:, :, :, :3], vae_tile_size)) + ref_imgs = [reference_image_0,reference_image_1,reference_image_2,reference_image_3,reference_image_4,reference_image_5,reference_image_6,reference_image_7] + for imgs in ref_imgs: + if imgs is None: continue + for i in range(imgs.shape[0]): + img = _resize_long_edge(imgs[i:i+1], ref_max_size, method=scale_method) + context.append(_encode_vae(vae, img[:,:,:,:3], vae_tile_size)) + if context: + positive = node_helpers.conditioning_set_values(positive, {"context_latents": context}) + negative = node_helpers.conditioning_set_values(negative, {"context_latents": context}) return (positive, negative, {"samples": latent}) \ No newline at end of file diff --git a/prestartup_script.py b/prestartup_script.py index 69330df..726ddd6 100644 --- a/prestartup_script.py +++ b/prestartup_script.py @@ -6,6 +6,7 @@ ComfyUI 启动时自动运行，清理旧版本残留文件。 """ import os +import shutil _HERE = os.path.dirname(os.path.abspath(__file__)) @@ -14,6 +15,18 @@ NEED_CLEAN = [ # 示例（将来迁移时取消注释）： # "nodes_minimax_h3_ref.py", # "js/xb_minimax_h3_ref.js", + + # 「机智罗的工作流」收拢进 jzl_workflows/ 之前的旧位置 + "xb_templates.py", + "js/xb_workflow_gallery.js", + "js/xb_workflow_gallery.css", + "js/xb_icon_dark.png", + "js/xb_icon_light.png", +] + +# 需要整目录删除的旧路径 +NEED_CLEAN_DIRS = [ + "xb_templates", ] for rel_path in NEED_CLEAN: @@ -22,8 +35,13 @@ for rel_path in NEED_CLEAN: os.remove(full) print(f"[XB_ToolBox] 清理旧文件: {rel_path}") +for rel_dir in NEED_CLEAN_DIRS: + full_dir = os.path.join(_HERE, rel_dir) + if os.path.isdir(full_dir): + shutil.rmtree(full_dir, ignore_errors=True) + print(f"[XB_ToolBox] 清理旧目录: {rel_dir}") + # 清理可能残留的旧 __pycache__ -import shutil for root, dirs, files in os.walk(_HERE): for d in dirs: if d == "__pycache__": diff --git a/tests/test_video_audio_presence.py b/tests/test_video_audio_presence.py index 52a88b4..24ff902 100644 --- a/tests/test_video_audio_presence.py +++ b/tests/test_video_audio_presence.py @@ -17,7 +17,7 @@ import torch def load_audio_function(): # Avoid importing unrelated ComfyUI/custom-node dependencies during CPU tests. - source = Path(__file__).resolve().parents[1] / 'nodes_video.py' + source = Path(__file__).resolve().parents[1] / 'nodes' / 'video' / 'nodes_video.py' tree = ast.parse(source.read_text(encoding='utf-8')) function = next(n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == '_lazy_get_audio') diff --git a/_add_io_mode_locale.py b/tools/_add_io_mode_locale.py similarity index 100% rename from _add_io_mode_locale.py rename to tools/_add_io_mode_locale.py diff --git a/_add_safe_print.py b/tools/_add_safe_print.py similarity index 100% rename from _add_safe_print.py rename to tools/_add_safe_print.py diff --git a/_add_skill_mode_locale.py b/tools/_add_skill_mode_locale.py similarity index 100% rename from _add_skill_mode_locale.py rename to tools/_add_skill_mode_locale.py diff --git a/_dump_clause.py b/tools/_dump_clause.py similarity index 100% rename from _dump_clause.py rename to tools/_dump_clause.py diff --git a/_dump_clause2.py b/tools/_dump_clause2.py similarity index 100% rename from _dump_clause2.py rename to tools/_dump_clause2.py diff --git a/_gen_pro_panel.py b/tools/_gen_pro_panel.py similarity index 97% rename from _gen_pro_panel.py rename to tools/_gen_pro_panel.py index afc4b3a..9a01db3 100644 --- a/_gen_pro_panel.py +++ b/tools/_gen_pro_panel.py @@ -9,10 +9,11 @@ import os import re import sys -ROOT = os.path.dirname(os.path.abspath(__file__)) -SRC = os.path.join(ROOT, "js", "xb_image_prompt_preset.js") -BLOCK = os.path.join(ROOT, "_xbr_pro_block.js") -DST = os.path.join(ROOT, "js", "xb_image_prompt_preset_pro.js") +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +_HERE = os.path.dirname(os.path.abspath(__file__)) +SRC = os.path.join(ROOT, "web", "xb_image_prompt_preset.js") +BLOCK = os.path.join(_HERE, "_xbr_pro_block.js") +DST = os.path.join(ROOT, "web", "xb_image_prompt_preset_pro.js") src = io.open(SRC, encoding="utf-8").read() block = io.open(BLOCK, encoding="utf-8").read() diff --git a/_preset_extract.py b/tools/_preset_extract.py similarity index 100% rename from _preset_extract.py rename to tools/_preset_extract.py diff --git a/_scan_spaces.py b/tools/_scan_spaces.py similarity index 100% rename from _scan_spaces.py rename to tools/_scan_spaces.py diff --git a/_xbr_pro_block.js b/tools/_xbr_pro_block.js similarity index 100% rename from _xbr_pro_block.js rename to tools/_xbr_pro_block.js diff --git a/js/assets/notify.mp3 b/web/assets/notify.mp3 similarity index 100% rename from js/assets/notify.mp3 rename to web/assets/notify.mp3 diff --git a/web/jzl_workflows/xb_workflow_gallery.js b/web/jzl_workflows/xb_workflow_gallery.js new file mode 100644 index 0000000..09eb282 --- /dev/null +++ b/web/jzl_workflows/xb_workflow_gallery.js @@ -0,0 +1,664 @@ +/** + * XB_ToolBox — 「机智罗的工作流」入口 + * ============================================================ + * 交互对齐官方「模板」： + * 点左侧栏的 JZL 图标 → 屏幕正中弹出大面板（左侧分类 + 顶部搜索 + 卡片网格）， + * 点卡片在工作流新标签页打开，Esc / 点遮罩 / 关闭按钮收起。 + * 侧栏 tab 只当作按钮用：弹出面板后立刻把侧栏收起，避免出现窄侧栏界面。 + * + * 数据：插件内 xb_templates/ 下的 index.json / index.zh.json + * 结构与官方 comfyui-workflow-templates 完全一致，按 category / title 分组。 + * 工作流本体与预览图由 xb_templates.py 托管，只放行清单登记过的文件。 + */ + +import { app } from "../../../scripts/app.js"; +import { api } from "../../../scripts/api.js"; + +const TAB_ID = "xb_workflow_gallery"; +const ASSET_BASE = "/xb_toolbox/templates/"; +const WEB_BASE = "/xb_toolbox/assets/"; +const ICON_CLASS = "xb-gallery-icon"; + +const MANIFESTS = { + zh: ["index.zh.json", "index.json"], + en: ["index.json", "index.zh.json"], +}; + +const STRINGS = { + zh: { + tab: "机智罗", + tooltip: "机智罗的工作流 · XB_ToolBox", + panel: "机智罗的工作流", + all: "全部", + search: "搜索工作流…", + refresh: "刷新", + close: "关闭", + loading: "正在加载工作流…", + empty: "还没有工作流", + noMatch: "没有匹配的工作流", + listFailed: "工作流清单加载失败", + loadFailed: "工作流加载失败", + openedNew: "已在新标签页打开", + loadedCurrent: "已加载到当前画布", + }, + en: { + tab: "JZL Workflows", + tooltip: "JZL Workflows · XB_ToolBox", + panel: "JZL Workflows", + all: "Show all", + search: "Search workflows…", + refresh: "Refresh", + close: "Close", + loading: "Loading workflows…", + empty: "No workflows yet", + noMatch: "No matching workflows", + listFailed: "Failed to load workflow index", + loadFailed: "Failed to load workflow", + openedNew: "Opened in a new workflow tab", + loadedCurrent: "Loaded into the current canvas", + }, +}; + +function currentLocale() { + try { + const value = app?.ui?.settings?.getSettingValue?.("Comfy.Locale"); + if (typeof value === "string" && value.toLowerCase().startsWith("zh")) return "zh"; + } catch (_) { /* ignore */ } + return "en"; +} + +function strings() { + return STRINGS[currentLocale()] || STRINGS.en; +} + +function el(tag, className, text) { + const node = document.createElement(tag); + if (className) node.className = className; + if (text !== undefined && text !== null) node.textContent = text; + return node; +} + +function toast(message, severity = "info") { + try { + const bus = app.extensionManager?.toast; + if (bus?.add) { + bus.add({ severity, summary: "XB_ToolBox", detail: message, life: 3000 }); + return; + } + } catch (_) { /* ignore */ } + console.log("[XB_ToolBox]", message); +} + +/** 当前主题是否为浅色（用于选中深浅两套图标） */ +function isLightTheme() { + try { + const palette = app?.ui?.settings?.getSettingValue?.("Comfy.ColorPalette"); + if (palette === "light") return true; + if (palette === "dark") return false; + } catch (_) { /* ignore */ } + try { + const match = getComputedStyle(document.body).backgroundColor.match(/\d+(?:\.\d+)?/g); + if (match && match.length >= 3) { + const [r, g, b] = match.slice(0, 3).map(Number); + return 0.299 * r + 0.587 * g + 0.114 * b > 140; + } + } catch (_) { /* ignore */ } + return false; +} + +/** 图标 / 样式走自己的 no-cache 路由，换图后普通刷新即可生效 */ +function webAsset(name) { + return api.apiURL(WEB_BASE + name); +} + +function iconUrl(name) { + return webAsset(name); +} + +function logoUrl() { + return iconUrl(isLightTheme() ? "xb_icon_light.png" : "xb_icon_dark.png"); +} + +async function fetchManifest() { + const order = MANIFESTS[currentLocale()] || MANIFESTS.en; + for (const name of order) { + try { + const response = await api.fetchApi(ASSET_BASE + name, { cache: "no-store" }); + if (!response.ok) continue; + const data = await response.json(); + if (Array.isArray(data)) return data; + } catch (error) { + console.warn(`[XB_ToolBox] 读取 ${name} 失败`, error); + } + } + return null; +} + +function flatten(manifest) { + const items = []; + for (const group of manifest) { + if (!group || !Array.isArray(group.templates)) continue; + const category = typeof group.category === "string" ? group.category : ""; + const subcategory = typeof group.title === "string" ? group.title : ""; + for (const tpl of group.templates) { + if (!tpl || typeof tpl.name !== "string" || !tpl.name) continue; + const thumbnails = Array.isArray(tpl.thumbnail) ? tpl.thumbnail : []; + const thumbnail = thumbnails.find((value) => typeof value === "string" && value) || ""; + items.push({ + name: tpl.name, + title: typeof tpl.title === "string" ? tpl.title : tpl.name, + description: typeof tpl.description === "string" ? tpl.description : "", + tags: Array.isArray(tpl.tags) ? tpl.tags.filter((tag) => typeof tag === "string") : [], + category, + subcategory, + thumbnail: thumbnail ? thumbnail.replace(/\\/g, "/").split("/").pop() : "", + }); + } + } + return items; +} + +function filterItems(items, subcategory, query) { + const needle = (query || "").trim().toLowerCase(); + return items.filter((item) => { + if (subcategory && (item.subcategory || item.category) !== subcategory) return false; + if (!needle) return true; + const haystack = [item.title, item.name, item.description, ...item.tags].join(" ").toLowerCase(); + return haystack.includes(needle); + }); +} + +function subcategoriesOf(items) { + const seen = []; + for (const item of items) { + const key = item.subcategory || item.category; + if (key && !seen.includes(key)) seen.push(key); + } + return seen; +} + +/** + * 新建空白工作流标签页。 + * 无 workflow 标签页机制的老版本会退化为清空当前画布，因此这里只做「尝试」。 + */ +async function tryNewBlankWorkflow() { + try { + const command = app.extensionManager?.command; + if (command?.execute) { + await command.execute("Comfy.NewBlankWorkflow"); + return true; + } + } catch (error) { + console.warn("[XB_ToolBox] 新建空白工作流失败，回退为替换当前画布", error); + } + return false; +} + +/** 打开清单里的一个工作流；返回是否新建了标签页 */ +async function openWorkflowItem(item) { + const response = await api.fetchApi(ASSET_BASE + item.name + ".json", { cache: "no-store" }); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + const graph = await response.json(); + + const workflowStore = app.extensionManager?.workflow; + const before = new Set(workflowStore?.openWorkflows?.map((wf) => wf.path) ?? []); + + // 工作流自带 id 时 loadGraphData 会直接开新标签页；否则先补一个空白标签页， + // 避免顶掉用户当前画布。 + if (!graph?.id && !graph?.path) await tryNewBlankWorkflow(); + + await app.loadGraphData(graph); + + // 模板不是用户保存过的文件，新标签页默认叫 “Unsaved Workflow (n)”，改成可读标题 + const created = workflowStore?.openWorkflows?.find((wf) => !before.has(wf.path)); + if (created && workflowStore.renameWorkflow) { + try { + await workflowStore.renameWorkflow(created, item.title); + } catch (error) { + console.warn("[XB_ToolBox] 重命名新标签页失败", error); + } + } + return Boolean(created); +} + +function buildCard(item, onOpen) { + const card = el("button", "xb-wf-card"); + card.type = "button"; + card.title = item.description || item.title; + + const thumb = el("div", "xb-wf-thumb"); + if (item.thumbnail) { + const img = document.createElement("img"); + img.loading = "lazy"; + img.alt = item.title; + img.src = api.apiURL(ASSET_BASE + item.thumbnail); + img.addEventListener("error", () => { + img.remove(); + thumb.classList.add("is-empty"); + thumb.append(el("span", "xb-wf-thumb-text", item.title)); + }); + thumb.append(img); + } else { + thumb.classList.add("is-empty"); + thumb.append(el("span", "xb-wf-thumb-text", item.title)); + } + + const name = el("div", "xb-wf-name", item.title); + const tags = el("div", "xb-wf-tags"); + for (const tag of item.tags.slice(0, 3)) tags.append(el("span", "xb-wf-tag", tag)); + + card.append(thumb, name, tags); + card.addEventListener("click", () => onOpen(item)); + return card; +} + +/** 居中大弹窗 —— 与官方「模板」一致的布局：左分类 / 顶部搜索 / 卡片网格 */ +class WorkflowModal { + constructor() { + this.items = []; + this.subcategory = ""; + this.query = ""; + this.busy = false; + } + + show() { + const text = strings(); + this.backdrop = el("div", "xb-wf-backdrop"); + this.backdrop.tabIndex = -1; + + const modal = el("div", "xb-wf-modal"); + modal.setAttribute("role", "dialog"); + modal.setAttribute("aria-modal", "true"); + modal.setAttribute("aria-label", text.panel); + + const side = el("aside", "xb-wf-side"); + const sideHead = el("div", "xb-wf-side-head"); + const logo = document.createElement("img"); + logo.className = "xb-wf-logo"; + logo.alt = ""; + logo.src = logoUrl(); + sideHead.append(logo, el("span", "xb-wf-side-title", text.panel)); + this.sideList = el("div", "xb-wf-side-list"); + side.append(sideHead, this.sideList); + + const main = el("section", "xb-wf-main"); + const head = el("header", "xb-wf-main-head"); + this.searchInput = document.createElement("input"); + this.searchInput.type = "search"; + this.searchInput.className = "xb-wf-search-input"; + this.searchInput.placeholder = text.search; + this.searchInput.addEventListener("input", () => { + this.query = this.searchInput.value; + this.renderGrid(); + }); + + this.refreshBtn = el("button", "xb-wf-icon-btn", "⟳"); + this.refreshBtn.type = "button"; + this.refreshBtn.title = text.refresh; + this.refreshBtn.addEventListener("click", () => this.load()); + + this.closeBtn = el("button", "xb-wf-icon-btn xb-wf-close", "✕"); + this.closeBtn.type = "button"; + this.closeBtn.title = text.close; + this.closeBtn.setAttribute("aria-label", text.close); + this.closeBtn.addEventListener("click", () => this.close()); + + head.append(this.searchInput, this.refreshBtn, this.closeBtn); + + this.grid = el("div", "xb-wf-grid"); + this.status = el("div", "xb-wf-status"); + main.append(head, this.grid, this.status); + + modal.append(side, main); + this.backdrop.append(modal); + document.body.append(this.backdrop); + + this.onKeyDown = (event) => { + if (event.key === "Escape") { + event.preventDefault(); + event.stopPropagation(); + this.close(); + } + }; + this.onBackdropClick = (event) => { + if (event.target === this.backdrop) this.close(); + }; + document.addEventListener("keydown", this.onKeyDown, true); + this.backdrop.addEventListener("mousedown", this.onBackdropClick); + + this.previousOverflow = document.body.style.overflow; + document.body.style.overflow = "hidden"; + + this.searchInput.focus(); + this.load(); + } + + setStatus(message, isError = false) { + this.status.textContent = message || ""; + this.status.classList.toggle("is-error", Boolean(isError)); + } + + async load() { + const text = strings(); + this.setStatus(text.loading); + this.refreshBtn.disabled = true; + try { + const manifest = await fetchManifest(); + if (manifest === null) { + this.items = []; + this.setStatus(text.listFailed, true); + } else { + this.items = flatten(manifest); + this.setStatus(""); + } + } catch (error) { + console.error("[XB_ToolBox] 工作流清单加载失败", error); + this.items = []; + this.setStatus(text.listFailed, true); + } finally { + this.refreshBtn.disabled = false; + this.renderCategories(); + this.renderGrid(); + } + } + + renderCategories() { + const text = strings(); + this.sideList.innerHTML = ""; + + const makeItem = (label, value) => { + const item = el("button", "xb-wf-side-item", label); + item.type = "button"; + item.classList.toggle("is-active", this.subcategory === value); + item.addEventListener("click", () => { + this.subcategory = value; + this.renderCategories(); + this.renderGrid(); + }); + return item; + }; + + this.sideList.append(makeItem(text.all, "")); + for (const name of subcategoriesOf(this.items)) this.sideList.append(makeItem(name, name)); + } + + renderGrid() { + const text = strings(); + this.grid.innerHTML = ""; + if (this.items.length === 0) { + this.grid.append(el("div", "xb-wf-empty", text.empty)); + return; + } + const visible = filterItems(this.items, this.subcategory, this.query); + if (visible.length === 0) { + this.grid.append(el("div", "xb-wf-empty", text.noMatch)); + return; + } + for (const item of visible) this.grid.append(buildCard(item, (target) => this.open(target))); + } + + async open(item) { + if (this.busy) return; + this.busy = true; + const text = strings(); + this.backdrop.classList.add("is-busy"); + try { + await openWorkflowItem(item); + toast(text.openedNew); + this.close(); + } catch (error) { + console.error("[XB_ToolBox] 工作流加载失败", error); + toast(`${text.loadFailed}${error?.message ? `: ${error.message}` : ""}`, "error"); + } finally { + this.busy = false; + this.backdrop?.classList.remove("is-busy"); + } + } + + close() { + if (!this.backdrop) return; + document.removeEventListener("keydown", this.onKeyDown, true); + this.backdrop.removeEventListener("mousedown", this.onBackdropClick); + this.backdrop.remove(); + document.body.style.overflow = this.previousOverflow || ""; + this.backdrop = null; + activeModal = null; + } +} + +/** 侧栏内联面板（弹窗创建失败时的兜底） */ +class WorkflowPanel { + constructor(host) { + this.host = host; + this.items = []; + this.subcategory = ""; + this.query = ""; + this.busy = false; + this.build(); + this.load(); + } + + build() { + const text = strings(); + this.host.classList.add("xb-wf-root"); + this.host.innerHTML = ""; + + const head = el("div", "xb-wf-head"); + const logo = document.createElement("img"); + logo.className = "xb-wf-logo"; + logo.alt = ""; + logo.src = logoUrl(); + this.refreshBtn = el("button", "xb-wf-icon-btn", "⟳"); + this.refreshBtn.type = "button"; + this.refreshBtn.title = text.refresh; + this.refreshBtn.addEventListener("click", () => this.load()); + head.append(logo, el("span", "xb-wf-title", text.panel), this.refreshBtn); + + const searchWrap = el("div", "xb-wf-search"); + this.searchInput = document.createElement("input"); + this.searchInput.type = "search"; + this.searchInput.placeholder = text.search; + this.searchInput.addEventListener("input", () => { + this.query = this.searchInput.value; + this.renderGrid(); + }); + searchWrap.append(this.searchInput); + + this.catBar = el("div", "xb-wf-cats"); + this.grid = el("div", "xb-wf-grid"); + this.status = el("div", "xb-wf-status"); + + this.host.append(head, searchWrap, this.catBar, this.grid, this.status); + } + + setStatus(message, isError = false) { + this.status.textContent = message || ""; + this.status.classList.toggle("is-error", Boolean(isError)); + } + + async load() { + const text = strings(); + this.setStatus(text.loading); + this.refreshBtn.disabled = true; + try { + const manifest = await fetchManifest(); + if (manifest === null) { + this.items = []; + this.setStatus(text.listFailed, true); + } else { + this.items = flatten(manifest); + this.setStatus(""); + } + } catch (error) { + console.error("[XB_ToolBox] 工作流清单加载失败", error); + this.items = []; + this.setStatus(text.listFailed, true); + } finally { + this.refreshBtn.disabled = false; + this.renderCategories(); + this.renderGrid(); + } + } + + renderCategories() { + const text = strings(); + this.catBar.innerHTML = ""; + const makeChip = (label, value) => { + const chip = el("button", "xb-wf-chip", label); + chip.type = "button"; + chip.classList.toggle("is-active", this.subcategory === value); + chip.addEventListener("click", () => { + this.subcategory = value; + this.renderCategories(); + this.renderGrid(); + }); + return chip; + }; + this.catBar.append(makeChip(text.all, "")); + for (const name of subcategoriesOf(this.items)) this.catBar.append(makeChip(name, name)); + this.catBar.classList.toggle("is-hidden", this.items.length === 0); + } + + renderGrid() { + const text = strings(); + this.grid.innerHTML = ""; + if (this.items.length === 0) { + this.grid.append(el("div", "xb-wf-empty", text.empty)); + return; + } + const visible = filterItems(this.items, this.subcategory, this.query); + if (visible.length === 0) { + this.grid.append(el("div", "xb-wf-empty", text.noMatch)); + return; + } + for (const item of visible) this.grid.append(buildCard(item, (target) => this.open(target))); + } + + async open(item) { + if (this.busy) return; + this.busy = true; + const text = strings(); + this.host.classList.add("is-busy"); + try { + await openWorkflowItem(item); + toast(text.openedNew); + } catch (error) { + console.error("[XB_ToolBox] 工作流加载失败", error); + toast(`${text.loadFailed}${error?.message ? `: ${error.message}` : ""}`, "error"); + } finally { + this.busy = false; + this.host.classList.remove("is-busy"); + } + } +} + +let activeModal = null; + +function openGalleryModal() { + try { + if (activeModal?.backdrop) { + activeModal.searchInput?.focus(); + return true; + } + activeModal = new WorkflowModal(); + activeModal.show(); + return Boolean(activeModal.backdrop); + } catch (error) { + console.error("[XB_ToolBox] 打开工作流面板失败，改用侧栏内联面板", error); + activeModal = null; + return false; + } +} + +/** + * 内置「模板」固定渲染在所有自定义 tab 之后，所以只要把自己排到扩展 tab 的最后一位， + * 就能紧邻「模板」上方。先立即注册保证入口及时出现，再在其它扩展注册完成后归一化一次位置。 + * 注意用数组内搬移而不是 unregister + register —— 后者会重复注册命令并打警告。 + */ +function moveToEndLater(manager, attempt = 0) { + setTimeout(() => { + try { + const tabs = manager.sidebarTab?.sidebarTabs; + if (!Array.isArray(tabs) || tabs.length === 0) { + if (attempt < 3) moveToEndLater(manager, attempt + 1); + return; + } + const index = tabs.findIndex((tab) => tab?.id === TAB_ID); + if (index === -1 || index === tabs.length - 1) return; + const [self] = tabs.splice(index, 1); + tabs.push(self); + } catch (error) { + console.warn("[XB_ToolBox] 调整「机智罗」侧栏位置失败", error); + } + }, attempt === 0 ? 1200 : 800); +} + +/** 侧栏 tab 只当作按钮：弹出面板后立刻把侧栏收起 */ +function collapseSidebarSoon() { + setTimeout(() => { + try { + app.extensionManager?.command?.execute?.(`Workspace.ToggleSidebarTab.${TAB_ID}`); + } catch (error) { + console.warn("[XB_ToolBox] 收起侧栏失败", error); + } + }, 0); +} + +function installAssets() { + const cssHref = webAsset("xb_workflow_gallery.css"); + if (!document.querySelector("link[data-xb-wf-css]")) { + const link = document.createElement("link"); + link.rel = "stylesheet"; + link.href = cssHref; + link.dataset.xbWfCss = "1"; + document.head.append(link); + } + + if (!document.querySelector("style[data-xb-wf-icon]")) { + const iconHref = iconUrl("xb_icon_dark.png"); + const style = document.createElement("style"); + style.dataset.xbWfIcon = "1"; + // 用 alpha 蒙版 + currentColor，图标自动跟随明/暗主题与选中态颜色 + style.textContent = ` +.${ICON_CLASS}::before { + content: ""; + display: inline-block; + width: 1.15em; + height: 1.15em; + background-color: currentColor; + -webkit-mask: url("${iconHref}") center / contain no-repeat; + mask: url("${iconHref}") center / contain no-repeat; +}`; + document.head.append(style); + } +} + +app.registerExtension({ + name: "XB_ToolBox.WorkflowGallery", + async setup() { + installAssets(); + const manager = app.extensionManager; + if (!manager?.registerSidebarTab) { + console.warn("[XB_ToolBox] 当前前端不支持 registerSidebarTab，跳过「机智罗的工作流」入口"); + return; + } + const text = strings(); + const tab = { + id: TAB_ID, + icon: ICON_CLASS, + title: text.tab, + tooltip: text.tooltip, + type: "custom", + render(host) { + if (openGalleryModal()) { + collapseSidebarSoon(); + return; + } + Object.assign(host.style, { height: "100%", minHeight: "0", overflow: "hidden" }); + if (!host.querySelector(".xb-wf-root")) new WorkflowPanel(host); + }, + }; + manager.registerSidebarTab(tab); + moveToEndLater(manager); + }, +}); diff --git a/js/xb_animate_relay_ui.js b/web/xb_animate_relay_ui.js similarity index 98% rename from js/xb_animate_relay_ui.js rename to web/xb_animate_relay_ui.js index 60748f7..2377bc7 100644 --- a/js/xb_animate_relay_ui.js +++ b/web/xb_animate_relay_ui.js @@ -1,47 +1,47 @@ -import { app } from "../../scripts/app.js"; -import { installRelayLock } from "./xb_compat.js"; - -// ============================================================ -// Animate & SCAIL 无限接力点 — 独立参考图 锁定/解锁 UI -// XB_WanAnimate_RelayNode(_New) / XB_WanInfiniteTalk_RelayNode_MultiRef(_AllInOne) -// ============================================================ -// 【Nodes 2.0 适配说明】 -// · 原实现把 🔒 蒙版画在 onDrawForeground、点击拦截放在 onMouseDown、预览屏蔽放在 onDrawBackground， -// 这三个钩子在 Nodes 2.0（DOM 渲染）下都不会被调用 → 锁定态完全失效。 -// · 现统一走 installRelayLock：锁定态直接隐藏"参考图文件 / 上传按钮"等本地部件， -// 并显示一个 DOM widget 提示条（经典模式与 2.0 表现一致）。 - -const RELAY_LOCK_NODES = ["XB_WanAnimate_RelayNode", "XB_WanAnimate_RelayNode_New", "XB_WanInfiniteTalk_RelayNode_MultiRef", "XB_WanInfiniteTalk_RelayNode_AllInOne"]; - -const MODE_WIDGETS = ["use_local_ref_image", "start_image_mode"]; -const UNLOCKED_VALUE = "独立参考图"; - -app.registerExtension({ - name: "xiaobai.animate_scail_relay_lock_ui", - async beforeRegisterNodeDef(nodeType, nodeData, app) { - if (!RELAY_LOCK_NODES.includes(nodeData.name)) return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - // 找不到模式开关时按"锁定"处理（与旧实现保持一致）：默认走总线全局参考图 - const isLocked = (n) => { - let tw = (n.widgets || []).find(w => w.name === "use_local_ref_image"); - if (!tw) tw = (n.widgets || []).find(w => w.name === "start_image_mode"); - if (!tw) return true; - return tw.value !== UNLOCKED_VALUE; - }; - - installRelayLock(node, { - text: "🔒 参考图继承自总线全局图", - bannerName: "xb_animate_relay_banner", - names: ["ref_image_file", "image_upload", "upload"], - watch: MODE_WIDGETS, - hideImgs: true, - locked: isLocked, - }); - }; - }, -}); +import { app } from "../../scripts/app.js"; +import { installRelayLock } from "./xb_compat.js"; + +// ============================================================ +// Animate & SCAIL 无限接力点 — 独立参考图 锁定/解锁 UI +// XB_WanAnimate_RelayNode(_New) / XB_WanInfiniteTalk_RelayNode_MultiRef(_AllInOne) +// ============================================================ +// 【Nodes 2.0 适配说明】 +// · 原实现把 🔒 蒙版画在 onDrawForeground、点击拦截放在 onMouseDown、预览屏蔽放在 onDrawBackground， +// 这三个钩子在 Nodes 2.0（DOM 渲染）下都不会被调用 → 锁定态完全失效。 +// · 现统一走 installRelayLock：锁定态直接隐藏"参考图文件 / 上传按钮"等本地部件， +// 并显示一个 DOM widget 提示条（经典模式与 2.0 表现一致）。 + +const RELAY_LOCK_NODES = ["XB_WanAnimate_RelayNode", "XB_WanAnimate_RelayNode_New", "XB_WanInfiniteTalk_RelayNode_MultiRef", "XB_WanInfiniteTalk_RelayNode_AllInOne"]; + +const MODE_WIDGETS = ["use_local_ref_image", "start_image_mode"]; +const UNLOCKED_VALUE = "独立参考图"; + +app.registerExtension({ + name: "xiaobai.animate_scail_relay_lock_ui", + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (!RELAY_LOCK_NODES.includes(nodeData.name)) return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + // 找不到模式开关时按"锁定"处理（与旧实现保持一致）：默认走总线全局参考图 + const isLocked = (n) => { + let tw = (n.widgets || []).find(w => w.name === "use_local_ref_image"); + if (!tw) tw = (n.widgets || []).find(w => w.name === "start_image_mode"); + if (!tw) return true; + return tw.value !== UNLOCKED_VALUE; + }; + + installRelayLock(node, { + text: "🔒 参考图继承自总线全局图", + bannerName: "xb_animate_relay_banner", + names: ["ref_image_file", "image_upload", "upload"], + watch: MODE_WIDGETS, + hideImgs: true, + locked: isLocked, + }); + }; + }, +}); diff --git a/js/xb_audio_slicer.js b/web/xb_audio_slicer.js similarity index 97% rename from js/xb_audio_slicer.js rename to web/xb_audio_slicer.js index 6f55a79..afe7f91 100644 --- a/js/xb_audio_slicer.js +++ b/web/xb_audio_slicer.js @@ -1,244 +1,244 @@ -import { app } from "../../scripts/app.js"; -import { sizeDomWidget } from "./xb_compat.js"; -import { api } from "../../scripts/api.js"; - -// ============================================================ -// XB_AudioSlicer — 复刻 VHS 加载音频 + 结束时间 + 自动时长 -// ============================================================ - -app.registerExtension({ - name: "XB_ToolBox.AudioSlicer", - async beforeRegisterNodeDef(nodeType, nodeData) { - if (nodeData.name !== "XB_AudioSlicer") return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - const wAudio = node.widgets.find(w => w.name === "audio"); - const wFps = node.widgets.find(w => w.name === "fps"); - const wStart = node.widgets.find(w => w.name === "start_time"); - const wEnd = node.widgets.find(w => w.name === "end_time"); - const wDur = node.widgets.find(w => w.name === "duration_display"); - - // ============================================================ - // 1. 时长显示设为只读+绿色样式 - // ============================================================ - if (wDur) { - // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） - if (wDur.options) wDur.options.read_only = true; - setTimeout(() => { - const el = wDur.inputEl || wDur.element; - if (el) { - el.readOnly = true; - el.style.backgroundColor = "#1a1a1a"; - el.style.color = "#00E676"; - el.style.textAlign = "center"; - el.style.fontWeight = "bold"; - el.style.fontSize = "13px"; - } - }, 100); - } - - // ============================================================ - // 2. 上传按钮 (VHS addWidget 模式) - // ============================================================ - if (wAudio) { - const fileInput = document.createElement("input"); - Object.assign(fileInput, { - type: "file", - accept: "audio/*,video/*", - style: "display: none", - onchange: async () => { - if (!fileInput.files.length) return; - const file = fileInput.files[0]; - try { - const body = new FormData(); - body.append("image", file); - const resp = await api.fetchApi("/upload/image", { method: "POST", body }); - if (resp.status === 200) { - const data = await resp.json(); - const name = data.name || file.name; - if (!wAudio.options.values.includes(name)) - wAudio.options.values.push(name); - wAudio.value = name; - if (wAudio.callback) wAudio.callback(name); - } - } catch (err) { - console.error("[XB_AudioSlicer] 上传失败:", err); - } - } - }); - document.body.appendChild(fileInput); - node.onRemoved = (() => { - const orig = node.onRemoved; - return () => { fileInput?.remove(); orig?.apply(node); }; - })(); - - const uploadBtn = node.addWidget("button", "选择音频上传", "image", () => { - app.canvas.node_widget = null; - fileInput.click(); - }); - uploadBtn.options.serialize = false; - } - - // ============================================================ - // 3. 音频播放器 — 限制在 start_time ~ end_time 范围 - // ============================================================ - const playerContainer = document.createElement("div"); - Object.assign(playerContainer.style, { - width: "100%", padding: "4px 0", boxSizing: "border-box" - }); - - const audioEl = document.createElement("audio"); - audioEl.controls = true; - audioEl.style.cssText = "width:100%;height:36px;outline:none;"; - playerContainer.appendChild(audioEl); - - const domWidget = node.addDOMWidget("xb_player", "custom", playerContainer); - sizeDomWidget(node, domWidget, 52); - - // --- 扩展节点初始高度 --- - if (node.size[1] < 280) node.size[1] = 280; - - // --- 限制播放范围 --- - audioEl.addEventListener("loadedmetadata", () => { - const st = parseFloat(wStart?.value) || 0; - if (st > 0 && st < audioEl.duration) { - audioEl.currentTime = st; - } - // 首次加载时自动设置 end_time = 全长（对齐到帧） - if (audioEl.duration > 0) { - const fps = parseFloat(wFps?.value) || 25; - const et = parseFloat(wEnd?.value) || 0; - if (et <= 0 || et > audioEl.duration + 1) { - const r = Math.floor(audioEl.duration * fps) / fps; - wEnd.value = r; - if (wEnd.inputEl) wEnd.inputEl.value = r; - if (wEnd.element) wEnd.element.value = r; - } - } - updateDuration(); - }); - - // 播放时限制在范围内；到达 end_time 后自动暂停并重置到 start_time - audioEl.addEventListener("timeupdate", () => { - const et = parseFloat(wEnd?.value) || audioEl.duration; - if (audioEl.currentTime >= et) { - audioEl.pause(); - const st = parseFloat(wStart?.value) || 0; - audioEl.currentTime = st; // 重置到开头，下次点击播放从头开始 - } - }); - - // 禁止拖拽进度条超过 end_time 或低于 start_time - audioEl.addEventListener("seeking", () => { - const et = parseFloat(wEnd?.value) || audioEl.duration; - const st = parseFloat(wStart?.value) || 0; - if (audioEl.currentTime > et) audioEl.currentTime = et; - if (audioEl.currentTime < st) audioEl.currentTime = st; - }); - - // --- 更新音频源（同一文件不重复加载）--- - let _xb_lastAudioFile = null; - const updateAudioSrc = () => { - const f = wAudio?.value; - if (f && f !== "none") { - if (f === _xb_lastAudioFile && audioEl.src && audioEl.readyState > 0) { - playerContainer.style.display = ""; - return; // 同一文件，已在播放，不重置 - } - _xb_lastAudioFile = f; - audioEl.src = api.apiURL("/view?" + new URLSearchParams({ - filename: f, type: "input", t: Date.now() - })); - playerContainer.style.display = ""; - } else { - playerContainer.style.display = "none"; - } - }; - - // ============================================================ - // 4. 帧数自动计算 - // ============================================================ - const updateDuration = () => { - let s = parseFloat(wStart?.value) || 0; - let e = parseFloat(wEnd?.value) || 0; - const fps = parseFloat(wFps?.value) || 25; - const fDur = 1.0 / fps; - // 防止 end_time < start_time + 1帧 - if (e < s + fDur) { - e = s + fDur; - wEnd.value = e; - if (wEnd.inputEl) wEnd.inputEl.value = e; - if (wEnd.element) wEnd.element.value = e; - } - const durSec = Math.max(0, e - s); - const rawFrames = Math.round(durSec * fps); - const frames = Math.max(1, Math.floor((rawFrames + 2) / 4) * 4 + 1); - const txt = frames + " 帧 (" + durSec.toFixed(2) + "s)"; - if (wDur && wDur.value !== txt) { - wDur.value = txt; - if (wDur.inputEl) wDur.inputEl.value = txt; - if (wDur.element) wDur.element.value = txt; - } - }; - - // --- start/end 变化 → 同步播放器位置 --- - const onTimeWidgetChange = () => { - // 防止 end < start + 1帧 - let s = parseFloat(wStart?.value) || 0; - let e = parseFloat(wEnd?.value) || 0; - const fps = parseFloat(wFps?.value) || 25; - const fDur = 1.0 / fps; - if (e < s + fDur) { - e = s + fDur; - wEnd.value = e; - if (wEnd.inputEl) wEnd.inputEl.value = e; - if (wEnd.element) wEnd.element.value = e; - } - updateDuration(); - if (audioEl.readyState >= 1) - audioEl.currentTime = s; - }; - if (wStart) { - const origSCb = wStart.callback; - wStart.callback = function () { origSCb?.apply(this, arguments); onTimeWidgetChange(); }; - } - if (wEnd) { - const origECb = wEnd.callback; - wEnd.callback = function () { origECb?.apply(this, arguments); onTimeWidgetChange(); }; - } - - // ============================================================ - // 5. 音频选择变化 → 更新播放器源 - // ============================================================ - if (wAudio) { - const origAudioCb = wAudio.callback; - wAudio.callback = function () { - if (origAudioCb) origAudioCb.apply(this, arguments); - updateAudioSrc(); - updateDuration(); - }; - } - - // ============================================================ - // 6. 定时同步 + 初始加载 - // ============================================================ - node._xb_interval = setInterval(updateDuration, 100); - - setTimeout(() => { - updateAudioSrc(); - }, 400); - - node.onRemoved = (() => { - const orig = node.onRemoved; - return () => { - clearInterval(node._xb_interval); - orig?.apply(node); - }; - })(); - }; - } -}); +import { app } from "../../scripts/app.js"; +import { sizeDomWidget } from "./xb_compat.js"; +import { api } from "../../scripts/api.js"; + +// ============================================================ +// XB_AudioSlicer — 复刻 VHS 加载音频 + 结束时间 + 自动时长 +// ============================================================ + +app.registerExtension({ + name: "XB_ToolBox.AudioSlicer", + async beforeRegisterNodeDef(nodeType, nodeData) { + if (nodeData.name !== "XB_AudioSlicer") return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + const wAudio = node.widgets.find(w => w.name === "audio"); + const wFps = node.widgets.find(w => w.name === "fps"); + const wStart = node.widgets.find(w => w.name === "start_time"); + const wEnd = node.widgets.find(w => w.name === "end_time"); + const wDur = node.widgets.find(w => w.name === "duration_display"); + + // ============================================================ + // 1. 时长显示设为只读+绿色样式 + // ============================================================ + if (wDur) { + // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） + if (wDur.options) wDur.options.read_only = true; + setTimeout(() => { + const el = wDur.inputEl || wDur.element; + if (el) { + el.readOnly = true; + el.style.backgroundColor = "#1a1a1a"; + el.style.color = "#00E676"; + el.style.textAlign = "center"; + el.style.fontWeight = "bold"; + el.style.fontSize = "13px"; + } + }, 100); + } + + // ============================================================ + // 2. 上传按钮 (VHS addWidget 模式) + // ============================================================ + if (wAudio) { + const fileInput = document.createElement("input"); + Object.assign(fileInput, { + type: "file", + accept: "audio/*,video/*", + style: "display: none", + onchange: async () => { + if (!fileInput.files.length) return; + const file = fileInput.files[0]; + try { + const body = new FormData(); + body.append("image", file); + const resp = await api.fetchApi("/upload/image", { method: "POST", body }); + if (resp.status === 200) { + const data = await resp.json(); + const name = data.name || file.name; + if (!wAudio.options.values.includes(name)) + wAudio.options.values.push(name); + wAudio.value = name; + if (wAudio.callback) wAudio.callback(name); + } + } catch (err) { + console.error("[XB_AudioSlicer] 上传失败:", err); + } + } + }); + document.body.appendChild(fileInput); + node.onRemoved = (() => { + const orig = node.onRemoved; + return () => { fileInput?.remove(); orig?.apply(node); }; + })(); + + const uploadBtn = node.addWidget("button", "选择音频上传", "image", () => { + app.canvas.node_widget = null; + fileInput.click(); + }); + uploadBtn.options.serialize = false; + } + + // ============================================================ + // 3. 音频播放器 — 限制在 start_time ~ end_time 范围 + // ============================================================ + const playerContainer = document.createElement("div"); + Object.assign(playerContainer.style, { + width: "100%", padding: "4px 0", boxSizing: "border-box" + }); + + const audioEl = document.createElement("audio"); + audioEl.controls = true; + audioEl.style.cssText = "width:100%;height:36px;outline:none;"; + playerContainer.appendChild(audioEl); + + const domWidget = node.addDOMWidget("xb_player", "custom", playerContainer); + sizeDomWidget(node, domWidget, 52); + + // --- 扩展节点初始高度 --- + if (node.size[1] < 280) node.size[1] = 280; + + // --- 限制播放范围 --- + audioEl.addEventListener("loadedmetadata", () => { + const st = parseFloat(wStart?.value) || 0; + if (st > 0 && st < audioEl.duration) { + audioEl.currentTime = st; + } + // 首次加载时自动设置 end_time = 全长（对齐到帧） + if (audioEl.duration > 0) { + const fps = parseFloat(wFps?.value) || 25; + const et = parseFloat(wEnd?.value) || 0; + if (et <= 0 || et > audioEl.duration + 1) { + const r = Math.floor(audioEl.duration * fps) / fps; + wEnd.value = r; + if (wEnd.inputEl) wEnd.inputEl.value = r; + if (wEnd.element) wEnd.element.value = r; + } + } + updateDuration(); + }); + + // 播放时限制在范围内；到达 end_time 后自动暂停并重置到 start_time + audioEl.addEventListener("timeupdate", () => { + const et = parseFloat(wEnd?.value) || audioEl.duration; + if (audioEl.currentTime >= et) { + audioEl.pause(); + const st = parseFloat(wStart?.value) || 0; + audioEl.currentTime = st; // 重置到开头，下次点击播放从头开始 + } + }); + + // 禁止拖拽进度条超过 end_time 或低于 start_time + audioEl.addEventListener("seeking", () => { + const et = parseFloat(wEnd?.value) || audioEl.duration; + const st = parseFloat(wStart?.value) || 0; + if (audioEl.currentTime > et) audioEl.currentTime = et; + if (audioEl.currentTime < st) audioEl.currentTime = st; + }); + + // --- 更新音频源（同一文件不重复加载）--- + let _xb_lastAudioFile = null; + const updateAudioSrc = () => { + const f = wAudio?.value; + if (f && f !== "none") { + if (f === _xb_lastAudioFile && audioEl.src && audioEl.readyState > 0) { + playerContainer.style.display = ""; + return; // 同一文件，已在播放，不重置 + } + _xb_lastAudioFile = f; + audioEl.src = api.apiURL("/view?" + new URLSearchParams({ + filename: f, type: "input", t: Date.now() + })); + playerContainer.style.display = ""; + } else { + playerContainer.style.display = "none"; + } + }; + + // ============================================================ + // 4. 帧数自动计算 + // ============================================================ + const updateDuration = () => { + let s = parseFloat(wStart?.value) || 0; + let e = parseFloat(wEnd?.value) || 0; + const fps = parseFloat(wFps?.value) || 25; + const fDur = 1.0 / fps; + // 防止 end_time < start_time + 1帧 + if (e < s + fDur) { + e = s + fDur; + wEnd.value = e; + if (wEnd.inputEl) wEnd.inputEl.value = e; + if (wEnd.element) wEnd.element.value = e; + } + const durSec = Math.max(0, e - s); + const rawFrames = Math.round(durSec * fps); + const frames = Math.max(1, Math.floor((rawFrames + 2) / 4) * 4 + 1); + const txt = frames + " 帧 (" + durSec.toFixed(2) + "s)"; + if (wDur && wDur.value !== txt) { + wDur.value = txt; + if (wDur.inputEl) wDur.inputEl.value = txt; + if (wDur.element) wDur.element.value = txt; + } + }; + + // --- start/end 变化 → 同步播放器位置 --- + const onTimeWidgetChange = () => { + // 防止 end < start + 1帧 + let s = parseFloat(wStart?.value) || 0; + let e = parseFloat(wEnd?.value) || 0; + const fps = parseFloat(wFps?.value) || 25; + const fDur = 1.0 / fps; + if (e < s + fDur) { + e = s + fDur; + wEnd.value = e; + if (wEnd.inputEl) wEnd.inputEl.value = e; + if (wEnd.element) wEnd.element.value = e; + } + updateDuration(); + if (audioEl.readyState >= 1) + audioEl.currentTime = s; + }; + if (wStart) { + const origSCb = wStart.callback; + wStart.callback = function () { origSCb?.apply(this, arguments); onTimeWidgetChange(); }; + } + if (wEnd) { + const origECb = wEnd.callback; + wEnd.callback = function () { origECb?.apply(this, arguments); onTimeWidgetChange(); }; + } + + // ============================================================ + // 5. 音频选择变化 → 更新播放器源 + // ============================================================ + if (wAudio) { + const origAudioCb = wAudio.callback; + wAudio.callback = function () { + if (origAudioCb) origAudioCb.apply(this, arguments); + updateAudioSrc(); + updateDuration(); + }; + } + + // ============================================================ + // 6. 定时同步 + 初始加载 + // ============================================================ + node._xb_interval = setInterval(updateDuration, 100); + + setTimeout(() => { + updateAudioSrc(); + }, 400); + + node.onRemoved = (() => { + const orig = node.onRemoved; + return () => { + clearInterval(node._xb_interval); + orig?.apply(node); + }; + })(); + }; + } +}); diff --git a/js/xb_audio_slicer_v1.js b/web/xb_audio_slicer_v1.js similarity index 98% rename from js/xb_audio_slicer_v1.js rename to web/xb_audio_slicer_v1.js index dec0634..65edf6d 100644 --- a/js/xb_audio_slicer_v1.js +++ b/web/xb_audio_slicer_v1.js @@ -1,233 +1,233 @@ -import { app } from "../../scripts/app.js"; -import { sizeDomWidget } from "./xb_compat.js"; -import { api } from "../../scripts/api.js"; - -// ============================================================ -// XB_AudioSlicerV1 — 通用波形 + 边缘拖拽吸附 + 丝滑定位 -// ============================================================ - -const PAD = 10, HANDLE_HIT = 12; - -app.registerExtension({ - name: "XB_ToolBox.AudioSlicerV1", - async beforeRegisterNodeDef(nodeType, nodeData) { - if (nodeData.name !== "XB_AudioSlicerV1") return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - const wAudio = node.widgets.find(w => w.name === "audio"); - const wFps = node.widgets.find(w => w.name === "fps"); - const wStart = node.widgets.find(w => w.name === "start_time"); - const wEnd = node.widgets.find(w => w.name === "end_time"); - const wDur = node.widgets.find(w => w.name === "duration_display"); - - if (wDur) { - // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） - if (wDur.options) wDur.options.read_only = true; - setTimeout(() => { - const el = wDur.inputEl || wDur.element; - if (el) { - el.readOnly = true; - el.style.cssText = "background-color:#1a1a1a;color:#00E676;text-align:center;font-weight:bold;font-size:15px;min-width:120px;white-space:nowrap;overflow:hidden;"; - } - }, 100); - } - - if (wAudio) { - const fi = document.createElement("input"); - fi.type = "file"; fi.accept = "audio/*,video/*"; fi.style.display = "none"; - fi.onchange = async () => { - if (!fi.files.length) return; - const body = new FormData(); body.append("image", fi.files[0]); - const resp = await api.fetchApi("/upload/image", { method: "POST", body }); - if (resp.status === 200) { - const d = await resp.json(); const n = d.name || fi.files[0].name; - if (!wAudio.options.values.includes(n)) wAudio.options.values.push(n); - wAudio.value = n; if (wAudio.callback) wAudio.callback(n); - } - }; - document.body.appendChild(fi); - const btn = node.addWidget("button", "选择音频上传", "image", () => { app.canvas.node_widget = null; fi.click(); }); - btn.options.serialize = false; - const o1 = node.onRemoved; node.onRemoved = () => { fi?.remove(); o1?.apply(node); }; - } - - const ctr = document.createElement("div"); - Object.assign(ctr.style, { display:"flex", flexDirection:"column", gap:"6px", width:"100%", padding:"6px", boxSizing:"border-box", background:"#161616", borderRadius:"6px", color:"#ccc", fontFamily:"sans-serif", marginTop:"4px" }); - - const audioEl = document.createElement("audio"); - audioEl.controls = true; audioEl.style.cssText = "width:100%;height:34px;outline:none;"; - ctr.appendChild(audioEl); - - const canvas = document.createElement("canvas"); - canvas.style.cssText = "width:100%;height:40px;border-radius:4px;background:#111;cursor:pointer;display:block;"; - const dpr = window.devicePixelRatio || 1; - canvas.width = 1200 * dpr; canvas.height = 80 * dpr; - ctr.appendChild(canvas); - - const domWidget = node.addDOMWidget("xb2_ui", "custom", ctr); - // Nodes 2.0：DOM widget 高度走 computeLayoutSize/getMinHeight（经典模式仍用 computeSize） - sizeDomWidget(node, domWidget, 100); - if (node.size[1] < 270) node.size[1] = 270; - - let totalDur = 0, _lastFile = null, _pausing = false; - let peaks = null, drag = null; - - const getDur = () => totalDur || 10; - const getS = () => parseFloat(wStart?.value) || 0; - const getE = () => { const e = parseFloat(wEnd?.value); return e > 0 ? e : getDur(); }; - const mouseToCanvasX = (clientX) => { const rect = canvas.getBoundingClientRect(); return (clientX - rect.left) * (canvas.width / rect.width); }; - - const syncWidgets = () => { - let s = getS(), e = getE(); - const fps = parseFloat(wFps?.value) || 25, fDur = 1.0 / fps; - if (e < s + fDur) e = s + fDur; - const dur = getDur(); if (e > dur) e = dur; - if (wStart.value !== s) { wStart.value = s; if (wStart.inputEl) wStart.inputEl.value = s; if (wStart.element) wStart.element.value = s; } - if (wEnd.value !== e) { wEnd.value = e; if (wEnd.inputEl) wEnd.inputEl.value = e; if (wEnd.element) wEnd.element.value = e; } - if (wDur) { - const durSec = Math.max(0, e - s); - const frames = Math.max(1, Math.floor((Math.round(durSec * fps) + 2) / 4) * 4 + 1); - const txt = frames + " 帧 (" + durSec.toFixed(2) + "s)"; - if (wDur.value !== txt) { wDur.value = txt; if (wDur.inputEl) wDur.inputEl.value = txt; if (wDur.element) wDur.element.value = txt; } - } - }; - - const visMapper = () => { - const dur = getDur(), s = getS(), e = getE(), selW = (e - s) || 0.01; - const sVis = Math.max(0, s - 0.125 * selW), eVis = Math.min(dur, e + 0.125 * selW); - const visW = (eVis - sVis) || 0.01, wwCSS = (canvas.width / dpr) - PAD * 2; - return { sVis, eVis, visW, wwCSS, toCSS: (time) => PAD + ((time - sVis) / visW) * wwCSS, toTime: (cssX) => sVis + ((cssX - PAD) / wwCSS) * visW }; - }; - - const draw = () => { - const ctx = canvas.getContext("2d"), w = canvas.width, h = canvas.height; - ctx.setTransform(dpr, 0, 0, dpr, 0, 0); - const W = w / dpr, H = h / dpr; - ctx.clearRect(0, 0, W, H); ctx.fillStyle = "#111"; ctx.fillRect(0, 0, W, H); - - const dur = getDur(), s = getS(), e = getE(), vm = visMapper(), ww = vm.wwCSS, mid = H / 2; - - if (peaks && peaks.length > 1) { - const startIdx = Math.max(0, Math.floor((vm.sVis / dur) * peaks.length)); - const endIdx = Math.min(peaks.length - 1, Math.ceil((vm.eVis / dur) * peaks.length)); - const vp = peaks.slice(startIdx, endIdx + 1); - if (vp.length > 1) { - const DISPLAY_POINTS = Math.min(1000, Math.floor(ww * 1.5)); - let maxPoints = [], minPoints = []; - const step = vp.length / DISPLAY_POINTS; - for (let i = 0; i < DISPLAY_POINTS; i++) { - const idx = i * step, lo = Math.floor(idx), hi = Math.min(lo + 1, vp.length - 1), frac = idx - lo; - maxPoints.push(vp[lo][0] * (1 - frac) + vp[hi][0] * frac); minPoints.push(vp[lo][1] * (1 - frac) + vp[hi][1] * frac); - } - const absMax = Math.max(Math.max(...maxPoints, 0.001), Math.abs(Math.min(...minPoints, -0.001))); - const scale = (v) => (v / absMax) * (H * 0.48); - - const pathActive = new Path2D(), pathInactive = new Path2D(); - pathActive.moveTo(PAD, mid); pathInactive.moveTo(PAD, mid); - for (let i = 0; i < DISPLAY_POINTS; i++) { - const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; - if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(maxPoints[i])); else pathInactive.lineTo(x, mid - scale(maxPoints[i])); - } - for (let i = DISPLAY_POINTS - 1; i >= 0; i--) { - const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; - if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(minPoints[i])); else pathInactive.lineTo(x, mid - scale(minPoints[i])); - } - pathActive.closePath(); pathInactive.closePath(); - ctx.fillStyle = "rgba(100,100,100,0.3)"; ctx.fill(pathInactive); ctx.fillStyle = "#4FC3F7"; ctx.fill(pathActive); - } - } - - // --- 处理拖拽边框 --- - let gx = vm.toCSS(s), rx = vm.toCSS(e); - if (drag && drag.type === "start_line" && typeof drag.tempX === "number") gx = drag.tempX; - if (drag && drag.type === "end_line" && typeof drag.tempX === "number") rx = drag.tempX; - if (gx > rx) { const tmp = gx; gx = rx; rx = tmp; } - - ctx.fillStyle = "rgba(0,0,0,0.65)"; ctx.fillRect(PAD, 0, gx - PAD, H); ctx.fillRect(rx, 0, W - PAD - rx, H); - ctx.fillStyle = "#4CAF50"; ctx.fillRect(gx - 2, 0, 4, H); - ctx.fillStyle = "#F44336"; ctx.fillRect(rx - 2, 0, 4, H); - - // --- 播放指针 --- - if (audioEl.readyState >= 1 && audioEl.currentTime > 0) { - const px = vm.toCSS(audioEl.currentTime); - if (px > PAD && px < W - PAD) { ctx.fillStyle = "#FFF"; ctx.fillRect(px - 1, 0, 2, H); } - } - - ctx.fillStyle = "#fff"; ctx.font = "bold 11px sans-serif"; - ctx.fillText(s.toFixed(2) + "s", Math.max(PAD + 4, gx - 45), 18); - const ewt = ctx.measureText(e.toFixed(2) + "s").width; - ctx.fillText(e.toFixed(2) + "s", Math.min(W - PAD - ewt - 4, rx + 6), 18); - ctx.setTransform(1, 0, 0, 1, 0, 0); - }; - - // ============================================================ - // 事件引擎：恢复拖拽与吸附 - // ============================================================ - canvas.onmousedown = (ev) => { - if (totalDur <= 0) return; - const mx = mouseToCanvasX(ev.clientX) / dpr, vm = visMapper(); - const sx = vm.toCSS(getS()), ex = vm.toCSS(getE()); - drag = null; - if (Math.abs(mx - sx) < HANDLE_HIT) { drag = { type: "start_line" }; return; } - if (Math.abs(mx - ex) < HANDLE_HIT) { drag = { type: "end_line" }; return; } - - // 点击非边缘区 → 直接跳转播放指针 - const clickTime = vm.toTime(mx); - if (audioEl.readyState >= 1) audioEl.currentTime = Math.max(0, Math.min(totalDur, clickTime)); - }; - - window.addEventListener("mousemove", (ev) => { - const mx = mouseToCanvasX(ev.clientX) / dpr; - if (drag) { - drag.tempX = Math.max(PAD, Math.min(canvas.width / dpr - PAD, mx)); - } else { - const rect = canvas.getBoundingClientRect(); - if (ev.clientX >= rect.left && ev.clientX <= rect.right && ev.clientY >= rect.top && ev.clientY <= rect.bottom) { - const vm = visMapper(), sx = vm.toCSS(getS()), ex = vm.toCSS(getE()); - canvas.style.cursor = (Math.abs(mx - sx) < HANDLE_HIT || Math.abs(mx - ex) < HANDLE_HIT) ? "ew-resize" : "pointer"; - } - } - }); - - window.addEventListener("mouseup", () => { - if (drag && typeof drag.tempX === "number") { - const vm = visMapper(), newVal = vm.toTime(drag.tempX); - const fps = parseFloat(wFps?.value) || 25, snap = v => Math.round(v * fps) / fps; - if (drag.type === "start_line") wStart.value = snap(Math.max(0, Math.min(newVal, getE() - 0.1))); - if (drag.type === "end_line") wEnd.value = snap(Math.min(totalDur, Math.max(newVal, getS() + 0.1))); - syncWidgets(); if (audioEl.readyState >= 1) audioEl.currentTime = getS(); - } - drag = null; - }); - - // ============================================================ - // 核心逻辑 - // ============================================================ - const updateSrc = () => { const f = wAudio?.value; if (f && f !== "none" && f !== _lastFile) { _lastFile = f; totalDur = 0; audioEl.src = api.apiURL("/view?" + new URLSearchParams({ filename: f, type: "input", t: Date.now() })); } }; - audioEl.addEventListener("loadedmetadata", () => { totalDur = audioEl.duration; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(wStart?.value) || 0; const curEnd = parseFloat(wEnd?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > totalDur + 1 || Math.abs(curEnd - 10.0) < 0.001)) { wStart.value = 0; wEnd.value = Math.floor(totalDur * fps) / fps; } syncWidgets(); audioEl.currentTime = parseFloat(wStart?.value) || 0; }); - audioEl.addEventListener("play", () => { _pausing = false; const st = getS(), et = getE(); if (audioEl.currentTime < st || audioEl.currentTime >= et) audioEl.currentTime = st; }); - audioEl.addEventListener("timeupdate", () => { if (_pausing) return; if (audioEl.currentTime >= getE()) { _pausing = true; audioEl.pause(); audioEl.currentTime = getS(); setTimeout(() => { _pausing = false; }, 100); } }); - - const fetchPeaks = async (filename) => { - if (!filename || filename === "none") { peaks = null; return; } - try { const resp = await api.fetchApi("/xb_toolbox/audio_waveform", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ filename, num_peaks: 4000 }) }); if (resp.ok) { const data = await resp.json(); peaks = data.peaks; totalDur = data.duration || 10; } } catch (e) {} - }; - - if (wAudio) { const orig = wAudio.callback; wAudio.callback = function () { orig?.apply(this, arguments); updateSrc(); fetchPeaks(wAudio.value).then(() => { const curStart = parseFloat(wStart?.value) || 0; const curEnd = parseFloat(wEnd?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > totalDur + 1 || Math.abs(curEnd - 10.0) < 0.001)) { wStart.value = 0; if (totalDur > 0) { const fps = parseFloat(wFps?.value) || 25; wEnd.value = Math.floor(totalDur * fps) / fps; } } syncWidgets(); }); }; } - const onTimeChange = (resetPlayback) => { let s = getS(), e = getE(), dur = getDur(); if (e < s + 0.01) { e = Math.min(s + 0.01, dur); wEnd.value = e; } syncWidgets(); if (resetPlayback && audioEl.readyState >= 1) audioEl.currentTime = s; }; - if (wStart) { const orig = wStart.callback; wStart.callback = function () { orig?.apply(this, arguments); onTimeChange(true); }; } - if (wEnd) { const orig = wEnd.callback; wEnd.callback = function () { orig?.apply(this, arguments); onTimeChange(false); }; } - - let _running = true; - const renderLoop = () => { if (!_running) return; if (totalDur > 0) { syncWidgets(); draw(); } requestAnimationFrame(renderLoop); }; - renderLoop(); - - setTimeout(() => { updateSrc(); if (wAudio?.value && wAudio.value !== "none") fetchPeaks(wAudio.value); }, 400); - const o2 = node.onRemoved; node.onRemoved = () => { _running = false; o2?.apply(node); }; - }; - } +import { app } from "../../scripts/app.js"; +import { sizeDomWidget } from "./xb_compat.js"; +import { api } from "../../scripts/api.js"; + +// ============================================================ +// XB_AudioSlicerV1 — 通用波形 + 边缘拖拽吸附 + 丝滑定位 +// ============================================================ + +const PAD = 10, HANDLE_HIT = 12; + +app.registerExtension({ + name: "XB_ToolBox.AudioSlicerV1", + async beforeRegisterNodeDef(nodeType, nodeData) { + if (nodeData.name !== "XB_AudioSlicerV1") return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + const wAudio = node.widgets.find(w => w.name === "audio"); + const wFps = node.widgets.find(w => w.name === "fps"); + const wStart = node.widgets.find(w => w.name === "start_time"); + const wEnd = node.widgets.find(w => w.name === "end_time"); + const wDur = node.widgets.find(w => w.name === "duration_display"); + + if (wDur) { + // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） + if (wDur.options) wDur.options.read_only = true; + setTimeout(() => { + const el = wDur.inputEl || wDur.element; + if (el) { + el.readOnly = true; + el.style.cssText = "background-color:#1a1a1a;color:#00E676;text-align:center;font-weight:bold;font-size:15px;min-width:120px;white-space:nowrap;overflow:hidden;"; + } + }, 100); + } + + if (wAudio) { + const fi = document.createElement("input"); + fi.type = "file"; fi.accept = "audio/*,video/*"; fi.style.display = "none"; + fi.onchange = async () => { + if (!fi.files.length) return; + const body = new FormData(); body.append("image", fi.files[0]); + const resp = await api.fetchApi("/upload/image", { method: "POST", body }); + if (resp.status === 200) { + const d = await resp.json(); const n = d.name || fi.files[0].name; + if (!wAudio.options.values.includes(n)) wAudio.options.values.push(n); + wAudio.value = n; if (wAudio.callback) wAudio.callback(n); + } + }; + document.body.appendChild(fi); + const btn = node.addWidget("button", "选择音频上传", "image", () => { app.canvas.node_widget = null; fi.click(); }); + btn.options.serialize = false; + const o1 = node.onRemoved; node.onRemoved = () => { fi?.remove(); o1?.apply(node); }; + } + + const ctr = document.createElement("div"); + Object.assign(ctr.style, { display:"flex", flexDirection:"column", gap:"6px", width:"100%", padding:"6px", boxSizing:"border-box", background:"#161616", borderRadius:"6px", color:"#ccc", fontFamily:"sans-serif", marginTop:"4px" }); + + const audioEl = document.createElement("audio"); + audioEl.controls = true; audioEl.style.cssText = "width:100%;height:34px;outline:none;"; + ctr.appendChild(audioEl); + + const canvas = document.createElement("canvas"); + canvas.style.cssText = "width:100%;height:40px;border-radius:4px;background:#111;cursor:pointer;display:block;"; + const dpr = window.devicePixelRatio || 1; + canvas.width = 1200 * dpr; canvas.height = 80 * dpr; + ctr.appendChild(canvas); + + const domWidget = node.addDOMWidget("xb2_ui", "custom", ctr); + // Nodes 2.0：DOM widget 高度走 computeLayoutSize/getMinHeight（经典模式仍用 computeSize） + sizeDomWidget(node, domWidget, 100); + if (node.size[1] < 270) node.size[1] = 270; + + let totalDur = 0, _lastFile = null, _pausing = false; + let peaks = null, drag = null; + + const getDur = () => totalDur || 10; + const getS = () => parseFloat(wStart?.value) || 0; + const getE = () => { const e = parseFloat(wEnd?.value); return e > 0 ? e : getDur(); }; + const mouseToCanvasX = (clientX) => { const rect = canvas.getBoundingClientRect(); return (clientX - rect.left) * (canvas.width / rect.width); }; + + const syncWidgets = () => { + let s = getS(), e = getE(); + const fps = parseFloat(wFps?.value) || 25, fDur = 1.0 / fps; + if (e < s + fDur) e = s + fDur; + const dur = getDur(); if (e > dur) e = dur; + if (wStart.value !== s) { wStart.value = s; if (wStart.inputEl) wStart.inputEl.value = s; if (wStart.element) wStart.element.value = s; } + if (wEnd.value !== e) { wEnd.value = e; if (wEnd.inputEl) wEnd.inputEl.value = e; if (wEnd.element) wEnd.element.value = e; } + if (wDur) { + const durSec = Math.max(0, e - s); + const frames = Math.max(1, Math.floor((Math.round(durSec * fps) + 2) / 4) * 4 + 1); + const txt = frames + " 帧 (" + durSec.toFixed(2) + "s)"; + if (wDur.value !== txt) { wDur.value = txt; if (wDur.inputEl) wDur.inputEl.value = txt; if (wDur.element) wDur.element.value = txt; } + } + }; + + const visMapper = () => { + const dur = getDur(), s = getS(), e = getE(), selW = (e - s) || 0.01; + const sVis = Math.max(0, s - 0.125 * selW), eVis = Math.min(dur, e + 0.125 * selW); + const visW = (eVis - sVis) || 0.01, wwCSS = (canvas.width / dpr) - PAD * 2; + return { sVis, eVis, visW, wwCSS, toCSS: (time) => PAD + ((time - sVis) / visW) * wwCSS, toTime: (cssX) => sVis + ((cssX - PAD) / wwCSS) * visW }; + }; + + const draw = () => { + const ctx = canvas.getContext("2d"), w = canvas.width, h = canvas.height; + ctx.setTransform(dpr, 0, 0, dpr, 0, 0); + const W = w / dpr, H = h / dpr; + ctx.clearRect(0, 0, W, H); ctx.fillStyle = "#111"; ctx.fillRect(0, 0, W, H); + + const dur = getDur(), s = getS(), e = getE(), vm = visMapper(), ww = vm.wwCSS, mid = H / 2; + + if (peaks && peaks.length > 1) { + const startIdx = Math.max(0, Math.floor((vm.sVis / dur) * peaks.length)); + const endIdx = Math.min(peaks.length - 1, Math.ceil((vm.eVis / dur) * peaks.length)); + const vp = peaks.slice(startIdx, endIdx + 1); + if (vp.length > 1) { + const DISPLAY_POINTS = Math.min(1000, Math.floor(ww * 1.5)); + let maxPoints = [], minPoints = []; + const step = vp.length / DISPLAY_POINTS; + for (let i = 0; i < DISPLAY_POINTS; i++) { + const idx = i * step, lo = Math.floor(idx), hi = Math.min(lo + 1, vp.length - 1), frac = idx - lo; + maxPoints.push(vp[lo][0] * (1 - frac) + vp[hi][0] * frac); minPoints.push(vp[lo][1] * (1 - frac) + vp[hi][1] * frac); + } + const absMax = Math.max(Math.max(...maxPoints, 0.001), Math.abs(Math.min(...minPoints, -0.001))); + const scale = (v) => (v / absMax) * (H * 0.48); + + const pathActive = new Path2D(), pathInactive = new Path2D(); + pathActive.moveTo(PAD, mid); pathInactive.moveTo(PAD, mid); + for (let i = 0; i < DISPLAY_POINTS; i++) { + const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; + if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(maxPoints[i])); else pathInactive.lineTo(x, mid - scale(maxPoints[i])); + } + for (let i = DISPLAY_POINTS - 1; i >= 0; i--) { + const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; + if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(minPoints[i])); else pathInactive.lineTo(x, mid - scale(minPoints[i])); + } + pathActive.closePath(); pathInactive.closePath(); + ctx.fillStyle = "rgba(100,100,100,0.3)"; ctx.fill(pathInactive); ctx.fillStyle = "#4FC3F7"; ctx.fill(pathActive); + } + } + + // --- 处理拖拽边框 --- + let gx = vm.toCSS(s), rx = vm.toCSS(e); + if (drag && drag.type === "start_line" && typeof drag.tempX === "number") gx = drag.tempX; + if (drag && drag.type === "end_line" && typeof drag.tempX === "number") rx = drag.tempX; + if (gx > rx) { const tmp = gx; gx = rx; rx = tmp; } + + ctx.fillStyle = "rgba(0,0,0,0.65)"; ctx.fillRect(PAD, 0, gx - PAD, H); ctx.fillRect(rx, 0, W - PAD - rx, H); + ctx.fillStyle = "#4CAF50"; ctx.fillRect(gx - 2, 0, 4, H); + ctx.fillStyle = "#F44336"; ctx.fillRect(rx - 2, 0, 4, H); + + // --- 播放指针 --- + if (audioEl.readyState >= 1 && audioEl.currentTime > 0) { + const px = vm.toCSS(audioEl.currentTime); + if (px > PAD && px < W - PAD) { ctx.fillStyle = "#FFF"; ctx.fillRect(px - 1, 0, 2, H); } + } + + ctx.fillStyle = "#fff"; ctx.font = "bold 11px sans-serif"; + ctx.fillText(s.toFixed(2) + "s", Math.max(PAD + 4, gx - 45), 18); + const ewt = ctx.measureText(e.toFixed(2) + "s").width; + ctx.fillText(e.toFixed(2) + "s", Math.min(W - PAD - ewt - 4, rx + 6), 18); + ctx.setTransform(1, 0, 0, 1, 0, 0); + }; + + // ============================================================ + // 事件引擎：恢复拖拽与吸附 + // ============================================================ + canvas.onmousedown = (ev) => { + if (totalDur <= 0) return; + const mx = mouseToCanvasX(ev.clientX) / dpr, vm = visMapper(); + const sx = vm.toCSS(getS()), ex = vm.toCSS(getE()); + drag = null; + if (Math.abs(mx - sx) < HANDLE_HIT) { drag = { type: "start_line" }; return; } + if (Math.abs(mx - ex) < HANDLE_HIT) { drag = { type: "end_line" }; return; } + + // 点击非边缘区 → 直接跳转播放指针 + const clickTime = vm.toTime(mx); + if (audioEl.readyState >= 1) audioEl.currentTime = Math.max(0, Math.min(totalDur, clickTime)); + }; + + window.addEventListener("mousemove", (ev) => { + const mx = mouseToCanvasX(ev.clientX) / dpr; + if (drag) { + drag.tempX = Math.max(PAD, Math.min(canvas.width / dpr - PAD, mx)); + } else { + const rect = canvas.getBoundingClientRect(); + if (ev.clientX >= rect.left && ev.clientX <= rect.right && ev.clientY >= rect.top && ev.clientY <= rect.bottom) { + const vm = visMapper(), sx = vm.toCSS(getS()), ex = vm.toCSS(getE()); + canvas.style.cursor = (Math.abs(mx - sx) < HANDLE_HIT || Math.abs(mx - ex) < HANDLE_HIT) ? "ew-resize" : "pointer"; + } + } + }); + + window.addEventListener("mouseup", () => { + if (drag && typeof drag.tempX === "number") { + const vm = visMapper(), newVal = vm.toTime(drag.tempX); + const fps = parseFloat(wFps?.value) || 25, snap = v => Math.round(v * fps) / fps; + if (drag.type === "start_line") wStart.value = snap(Math.max(0, Math.min(newVal, getE() - 0.1))); + if (drag.type === "end_line") wEnd.value = snap(Math.min(totalDur, Math.max(newVal, getS() + 0.1))); + syncWidgets(); if (audioEl.readyState >= 1) audioEl.currentTime = getS(); + } + drag = null; + }); + + // ============================================================ + // 核心逻辑 + // ============================================================ + const updateSrc = () => { const f = wAudio?.value; if (f && f !== "none" && f !== _lastFile) { _lastFile = f; totalDur = 0; audioEl.src = api.apiURL("/view?" + new URLSearchParams({ filename: f, type: "input", t: Date.now() })); } }; + audioEl.addEventListener("loadedmetadata", () => { totalDur = audioEl.duration; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(wStart?.value) || 0; const curEnd = parseFloat(wEnd?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > totalDur + 1 || Math.abs(curEnd - 10.0) < 0.001)) { wStart.value = 0; wEnd.value = Math.floor(totalDur * fps) / fps; } syncWidgets(); audioEl.currentTime = parseFloat(wStart?.value) || 0; }); + audioEl.addEventListener("play", () => { _pausing = false; const st = getS(), et = getE(); if (audioEl.currentTime < st || audioEl.currentTime >= et) audioEl.currentTime = st; }); + audioEl.addEventListener("timeupdate", () => { if (_pausing) return; if (audioEl.currentTime >= getE()) { _pausing = true; audioEl.pause(); audioEl.currentTime = getS(); setTimeout(() => { _pausing = false; }, 100); } }); + + const fetchPeaks = async (filename) => { + if (!filename || filename === "none") { peaks = null; return; } + try { const resp = await api.fetchApi("/xb_toolbox/audio_waveform", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ filename, num_peaks: 4000 }) }); if (resp.ok) { const data = await resp.json(); peaks = data.peaks; totalDur = data.duration || 10; } } catch (e) {} + }; + + if (wAudio) { const orig = wAudio.callback; wAudio.callback = function () { orig?.apply(this, arguments); updateSrc(); fetchPeaks(wAudio.value).then(() => { const curStart = parseFloat(wStart?.value) || 0; const curEnd = parseFloat(wEnd?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > totalDur + 1 || Math.abs(curEnd - 10.0) < 0.001)) { wStart.value = 0; if (totalDur > 0) { const fps = parseFloat(wFps?.value) || 25; wEnd.value = Math.floor(totalDur * fps) / fps; } } syncWidgets(); }); }; } + const onTimeChange = (resetPlayback) => { let s = getS(), e = getE(), dur = getDur(); if (e < s + 0.01) { e = Math.min(s + 0.01, dur); wEnd.value = e; } syncWidgets(); if (resetPlayback && audioEl.readyState >= 1) audioEl.currentTime = s; }; + if (wStart) { const orig = wStart.callback; wStart.callback = function () { orig?.apply(this, arguments); onTimeChange(true); }; } + if (wEnd) { const orig = wEnd.callback; wEnd.callback = function () { orig?.apply(this, arguments); onTimeChange(false); }; } + + let _running = true; + const renderLoop = () => { if (!_running) return; if (totalDur > 0) { syncWidgets(); draw(); } requestAnimationFrame(renderLoop); }; + renderLoop(); + + setTimeout(() => { updateSrc(); if (wAudio?.value && wAudio.value !== "none") fetchPeaks(wAudio.value); }, 400); + const o2 = node.onRemoved; node.onRemoved = () => { _running = false; o2?.apply(node); }; + }; + } }); \ No newline at end of file diff --git a/js/xb_audio_slicer_v2.js b/web/xb_audio_slicer_v2.js similarity index 98% rename from js/xb_audio_slicer_v2.js rename to web/xb_audio_slicer_v2.js index 217190b..4e756a7 100644 --- a/js/xb_audio_slicer_v2.js +++ b/web/xb_audio_slicer_v2.js @@ -1,245 +1,245 @@ -import { app } from "../../scripts/app.js"; -import { sizeDomWidget } from "./xb_compat.js"; -import { api } from "../../scripts/api.js"; - -// ============================================================ -// XB_AudioSlicerV2 — 双人音频切片 + 边缘拖拽吸附 + 丝滑定位 -// ============================================================ - -const PAD = 10, HANDLE_HIT = 12; - -app.registerExtension({ - name: "XB_ToolBox.AudioSlicerV2", - async beforeRegisterNodeDef(nodeType, nodeData) { - if (nodeData.name !== "XB_AudioSlicerV2") return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - const wA1 = node.widgets.find(w => w.name === "audio1"); - const wS1 = node.widgets.find(w => w.name === "start1"); - const wE1 = node.widgets.find(w => w.name === "end1"); - const wA2 = node.widgets.find(w => w.name === "audio2"); - const wS2 = node.widgets.find(w => w.name === "start2"); - const wE2 = node.widgets.find(w => w.name === "end2"); - const wGap = node.widgets.find(w => w.name === "gap_frames"); - const wFps = node.widgets.find(w => w.name === "fps"); - const wTot = node.widgets.find(w => w.name === "total_display"); - - // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） - if (wTot && wTot.options) wTot.options.read_only = true; - if (wTot) setTimeout(() => { const el = wTot.inputEl || wTot.element; if (el) { el.readOnly = true; el.style.cssText = "background-color:#1a1a1a;color:#00E676;text-align:center;font-weight:bold;font-size:14px;min-width:120px;white-space:nowrap;overflow:hidden;"; } }, 100); - - const audioEl1 = document.createElement("audio"); audioEl1.controls = true; audioEl1.style.cssText = "width:100%;height:30px;outline:none;"; - const audioEl2 = document.createElement("audio"); audioEl2.controls = true; audioEl2.style.cssText = "width:100%;height:30px;outline:none;"; - let dur1 = 0, dur2 = 0, pausing = false, _last1 = null, _last2 = null; - let peaks1 = null, peaks2 = null; - let drags = { 1: null, 2: null }; // 存储双轨的拖动状态 - - const ctr = document.createElement("div"); - Object.assign(ctr.style, { display:"flex", flexDirection:"column", gap:"4px", width:"100%", padding:"6px", boxSizing:"border-box", background:"#161616", borderRadius:"6px", color:"#ccc", fontFamily:"sans-serif", marginTop:"4px" }); - - const mkUpload = (w, label) => { - const fi = document.createElement("input"); fi.type = "file"; fi.accept = "audio/*,video/*"; fi.style.display = "none"; - fi.onchange = async () => { - if (!fi.files.length) return; - const body = new FormData(); body.append("image", fi.files[0]); - const resp = await api.fetchApi("/upload/image", { method: "POST", body }); - if (resp.status === 200) { const d = await resp.json(); const n = d.name || fi.files[0].name; if (!w.options.values.includes(n)) w.options.values.push(n); w.value = n; if (w.callback) w.callback(n); } - }; - document.body.appendChild(fi); - const btn = node.addWidget("button", label, "image", () => { app.canvas.node_widget = null; fi.click(); }); - if (btn.options) btn.options.serialize = false; - return fi; - }; - mkUpload(wA1, "📁 上传音频 1"); mkUpload(wA2, "📁 上传音频 2"); - - const dpr = window.devicePixelRatio || 1; - const label1 = document.createElement("div"); label1.textContent = "🎵 音频1"; label1.style.cssText = "font-size:11px;color:#4FC3F7;font-weight:bold;"; - ctr.appendChild(label1); ctr.appendChild(audioEl1); - const canvas1 = document.createElement("canvas"); canvas1.style.cssText = "width:100%;height:36px;border-radius:3px;background:#111;cursor:pointer;display:block;"; - canvas1.width = 1200 * dpr; canvas1.height = 72 * dpr; ctr.appendChild(canvas1); - - const label2 = document.createElement("div"); label2.textContent = "🎵 音频2"; label2.style.cssText = "font-size:11px;color:#F44336;font-weight:bold;"; - ctr.appendChild(label2); ctr.appendChild(audioEl2); - const canvas2 = document.createElement("canvas"); canvas2.style.cssText = "width:100%;height:36px;border-radius:3px;background:#111;cursor:pointer;display:block;"; - canvas2.width = 1200 * dpr; canvas2.height = 72 * dpr; ctr.appendChild(canvas2); - - const domWidget = node.addDOMWidget("xb3_ui", "custom", ctr); - // Nodes 2.0：DOM widget 高度走 computeLayoutSize/getMinHeight（经典模式仍用 computeSize） - sizeDomWidget(node, domWidget, 240); - if (node.size[1] < 520) node.size[1] = 520; - - const getD = (t) => t === 1 ? (dur1 || 10) : (dur2 || 10); - const getS = (t) => t === 1 ? (parseFloat(wS1?.value) || 0) : (parseFloat(wS2?.value) || 0); - const getE = (t) => { const e = t === 1 ? (parseFloat(wE1?.value) || 0) : (parseFloat(wE2?.value) || 0); return e > 0 ? e : getD(t); }; - const mouseToX = (cx, cvs) => { const r = cvs.getBoundingClientRect(); return (cx - r.left) * (cvs.width / r.width); }; - - const syncW = (t) => { - const s = getS(t), e = getE(t), ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; - const dur = getD(t); if (e > dur) { if (we) we.value = dur; } - if (ws) { ws.value = s; if (ws.inputEl) ws.inputEl.value = s; if (ws.element) ws.element.value = s; } - if (we) { we.value = e; if (we.inputEl) we.inputEl.value = e; if (we.element) we.element.value = e; } - const durSec = Math.max(0, e - s); - const lbl = t === 1 ? label1 : label2; - lbl.textContent = (t === 1 ? "🎵 音频1" : "🎵 音频2") + " (已选择 " + durSec.toFixed(2) + "秒)"; - if (wTot) { - const fps = parseFloat(wFps?.value) || 25, g = parseInt(wGap?.value) || 0; - const d1 = Math.max(0, getE(1) - getS(1)), d2 = Math.max(0, getE(2) - getS(2)); - const totalFrames = Math.max(1, Math.floor(((Math.round(d1 * fps) + g + Math.round(d2 * fps)) + 2) / 4) * 4 + 1); - const tot = totalFrames + " 帧 (" + (d1 + g / fps + d2).toFixed(2) + "s)"; - if (wTot.value !== tot) { wTot.value = tot; if (wTot.inputEl) wTot.inputEl.value = tot; if (wTot.element) wTot.element.value = tot; } - } - }; - - const fetchPeaks = async (filename, track) => { - if (!filename || filename === "none") { if (track === 1) peaks1 = null; else peaks2 = null; return; } - try { - const resp = await api.fetchApi("/xb_toolbox/audio_waveform", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ filename, num_peaks: 4000 }) }); - if (resp.ok) { const data = await resp.json(); if (track === 1) { peaks1 = data.peaks; dur1 = data.duration; } else { peaks2 = data.peaks; dur2 = data.duration; } } - } catch (e) {} - }; - - const visMapper = (cvs, t) => { - const s = getS(t), e = getE(t), dur = getD(t), selW = (e - s) || 0.01; - const sVis = Math.max(0, s - 0.125 * selW), eVis = Math.min(dur, e + 0.125 * selW); - const visW = (eVis - sVis) || 0.01, wwCSS = (cvs.width / dpr) - PAD * 2; - return { sVis, eVis, visW, wwCSS, toCSS: (time) => PAD + ((time - sVis) / visW) * wwCSS, toTime: (cssX) => sVis + ((cssX - PAD) / wwCSS) * visW }; - }; - - const drawWave = (cvs, t) => { - const ctx = cvs.getContext("2d"), W = cvs.width / dpr, H = cvs.height / dpr; - ctx.setTransform(dpr, 0, 0, dpr, 0, 0); - ctx.clearRect(0, 0, W, H); ctx.fillStyle = "#111"; ctx.fillRect(0, 0, W, H); - const d = getD(t), s = getS(t), e = getE(t), vm = visMapper(cvs, t), ww = vm.wwCSS, mid = H / 2; - const peaks = t === 1 ? peaks1 : peaks2; - - if (peaks && peaks.length > 1) { - const startIdx = Math.max(0, Math.floor((vm.sVis / d) * peaks.length)), endIdx = Math.min(peaks.length - 1, Math.ceil((vm.eVis / d) * peaks.length)); - const vp = peaks.slice(startIdx, endIdx + 1); - if (vp.length > 1) { - const DISPLAY_POINTS = Math.min(1000, Math.floor((vm.wwCSS || ww) * 1.5)); - let maxPoints = [], minPoints = []; - const step = vp.length / DISPLAY_POINTS; - for (let i = 0; i < DISPLAY_POINTS; i++) { - const idx = i * step, lo = Math.floor(idx), hi = Math.min(lo + 1, vp.length - 1), frac = idx - lo; - maxPoints.push(vp[lo][0] * (1 - frac) + vp[hi][0] * frac); minPoints.push(vp[lo][1] * (1 - frac) + vp[hi][1] * frac); - } - const absMax = Math.max(Math.max(...maxPoints, 0.001), Math.abs(Math.min(...minPoints, -0.001))); - const scale = (v) => (v / absMax) * (H * 0.48); - - const pathActive = new Path2D(), pathInactive = new Path2D(); - pathActive.moveTo(PAD, mid); pathInactive.moveTo(PAD, mid); - for (let i = 0; i < DISPLAY_POINTS; i++) { - const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; - if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(maxPoints[i])); else pathInactive.lineTo(x, mid - scale(maxPoints[i])); - } - for (let i = DISPLAY_POINTS - 1; i >= 0; i--) { - const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; - if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(minPoints[i])); else pathInactive.lineTo(x, mid - scale(minPoints[i])); - } - pathActive.closePath(); pathInactive.closePath(); - ctx.fillStyle = "rgba(100,100,100,0.3)"; ctx.fill(pathInactive); ctx.fillStyle = t === 1 ? "#4FC3F7" : "#F44336"; ctx.fill(pathActive); - } - } - - // 处理拖拽边框 - let gx = vm.toCSS(s), rx = vm.toCSS(e); - if (drags[t] && drags[t].type === "start_line" && typeof drags[t].tempX === "number") gx = drags[t].tempX; - if (drags[t] && drags[t].type === "end_line" && typeof drags[t].tempX === "number") rx = drags[t].tempX; - if (gx > rx) { const tmp = gx; gx = rx; rx = tmp; } - - ctx.fillStyle = "rgba(0,0,0,0.65)"; ctx.fillRect(PAD, 0, gx - PAD, H); ctx.fillRect(rx, 0, W - PAD - rx, H); - ctx.fillStyle = "#4CAF50"; ctx.fillRect(gx - 2, 0, 4, H); - ctx.fillStyle = "#F44336"; ctx.fillRect(rx - 2, 0, 4, H); - - const ael = t === 1 ? audioEl1 : audioEl2; - if (ael.readyState >= 1 && ael.currentTime > 0) { - const px = vm.toCSS(ael.currentTime); - if (px > PAD && px < W - PAD) { ctx.fillStyle = "#FFF"; ctx.fillRect(px - 1, 0, 2, H); } - } - - ctx.fillStyle = "#fff"; ctx.font = "bold 10px sans-serif"; - ctx.fillText(s.toFixed(2) + "s", Math.max(PAD + 4, gx - 45), 14); - const ewt = ctx.measureText(e.toFixed(2) + "s").width; - ctx.fillText(e.toFixed(2) + "s", Math.min(W - PAD - ewt - 4, rx + 6), 14); - ctx.setTransform(1, 0, 0, 1, 0, 0); - }; - - // ============================================================ - // 鼠标事件引擎 - // ============================================================ - const bindMouse = (cvs, t) => { - cvs.onmousedown = (ev) => { - if (getD(t) <= 0) return; - const mx = mouseToX(ev.clientX, cvs) / dpr, vm = visMapper(cvs, t); - const sx = vm.toCSS(getS(t)), ex = vm.toCSS(getE(t)); - drags[t] = null; - if (Math.abs(mx - sx) < HANDLE_HIT) { drags[t] = { type: "start_line" }; return; } - if (Math.abs(mx - ex) < HANDLE_HIT) { drags[t] = { type: "end_line" }; return; } - - const clickTime = vm.toTime(mx), ael = t === 1 ? audioEl1 : audioEl2; - if (ael.readyState >= 1) ael.currentTime = Math.max(0, Math.min(getD(t), clickTime)); - }; - - window.addEventListener("mousemove", (ev) => { - const mx = mouseToX(ev.clientX, cvs) / dpr; - if (drags[t]) { - drags[t].tempX = Math.max(PAD, Math.min(cvs.width / dpr - PAD, mx)); - } else { - const rect = cvs.getBoundingClientRect(); - if (ev.clientX >= rect.left && ev.clientX <= rect.right && ev.clientY >= rect.top && ev.clientY <= rect.bottom) { - const vm = visMapper(cvs, t), sx = vm.toCSS(getS(t)), ex = vm.toCSS(getE(t)); - cvs.style.cursor = (Math.abs(mx - sx) < HANDLE_HIT || Math.abs(mx - ex) < HANDLE_HIT) ? "ew-resize" : "pointer"; - } - } - }); - }; - bindMouse(canvas1, 1); bindMouse(canvas2, 2); - - window.addEventListener("mouseup", () => { - [1, 2].forEach(t => { - if (drags[t] && typeof drags[t].tempX === "number") { - const cvs = t === 1 ? canvas1 : canvas2, vm = visMapper(cvs, t), newVal = vm.toTime(drags[t].tempX); - const fps = parseFloat(wFps?.value) || 25, snap = v => Math.round(v * fps) / fps; - const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; - if (drags[t].type === "start_line") ws.value = snap(Math.max(0, Math.min(newVal, getE(t) - 0.1))); - if (drags[t].type === "end_line") we.value = snap(Math.min(getD(t), Math.max(newVal, getS(t) + 0.1))); - syncW(t); const ael = t === 1 ? audioEl1 : audioEl2; if (ael.readyState >= 1) ael.currentTime = getS(t); - } - drags[t] = null; - }); - }); - - // ============================================================ - // 音频逻辑 - // ============================================================ - const updateSrc = (ael, w, t, _last) => { const f = w?.value; if (f && f !== "none" && f !== _last) { if (t === 1) _last1 = f; else _last2 = f; ael.src = api.apiURL("/view?" + new URLSearchParams({ filename: f, type: "input", t: Date.now() })); } }; - const bindAudio = (ael, t) => { - ael.addEventListener("loadedmetadata", () => { if (t === 1) dur1 = ael.duration; else dur2 = ael.duration; const d = getD(t); const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(ws?.value) || 0; const curEnd = parseFloat(we?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > d + 1 || Math.abs(curEnd - 10.0) < 0.001)) { ws.value = 0; we.value = Math.floor(d * fps) / fps; } syncW(t); ael.currentTime = parseFloat(ws?.value) || 0; }); - ael.addEventListener("play", () => { const s = getS(t); if (ael.currentTime < s || ael.currentTime >= getE(t)) ael.currentTime = s; }); - ael.addEventListener("timeupdate", () => { if (pausing) return; if (ael.currentTime >= getE(t)) { pausing = true; ael.pause(); ael.currentTime = getS(t); setTimeout(() => { pausing = false; }, 100); } }); - }; - bindAudio(audioEl1, 1); bindAudio(audioEl2, 2); - - const onTimeChange = (t, resetPlayback) => { syncW(t); if (resetPlayback) { const ael = t === 1 ? audioEl1 : audioEl2; if (ael.readyState >= 1) ael.currentTime = getS(t); } }; - - [wA1, wA2].forEach((w, i) => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); const t = i + 1; const fn = w.value; fetchPeaks(fn, t).then(() => { const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; const d = t === 1 ? dur1 : dur2; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(ws?.value) || 0; const curEnd = parseFloat(we?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > d + 1 || Math.abs(curEnd - 10.0) < 0.001)) { ws.value = 0; we.value = Math.floor(d * fps) / fps; } syncW(t); }); updateSrc(i === 0 ? audioEl1 : audioEl2, w, t, i === 0 ? _last1 : _last2); }; } }); - [wS1].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(1, true); }; } }); - [wE1].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(1, false); }; } }); - [wS2].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(2, true); }; } }); - [wE2].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(2, false); }; } }); - if (wGap) { const o = wGap.callback; wGap.callback = function () { o?.apply(this, arguments); syncW(1); syncW(2); }; } - - let _running = true; - const renderLoop = () => { if (!_running) return; if (dur1 > 0 || dur2 > 0) { syncW(1); syncW(2); drawWave(canvas1, 1); drawWave(canvas2, 2); } requestAnimationFrame(renderLoop); }; - renderLoop(); - - setTimeout(() => { updateSrc(audioEl1, wA1, 1, _last1); updateSrc(audioEl2, wA2, 2, _last2); if (wA1?.value && wA1.value !== "none") fetchPeaks(wA1.value, 1); if (wA2?.value && wA2.value !== "none") fetchPeaks(wA2.value, 2); }, 400); - - const oo = node.onRemoved; node.onRemoved = () => { _running = false; oo?.apply(node); }; - }; - } +import { app } from "../../scripts/app.js"; +import { sizeDomWidget } from "./xb_compat.js"; +import { api } from "../../scripts/api.js"; + +// ============================================================ +// XB_AudioSlicerV2 — 双人音频切片 + 边缘拖拽吸附 + 丝滑定位 +// ============================================================ + +const PAD = 10, HANDLE_HIT = 12; + +app.registerExtension({ + name: "XB_ToolBox.AudioSlicerV2", + async beforeRegisterNodeDef(nodeType, nodeData) { + if (nodeData.name !== "XB_AudioSlicerV2") return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + const wA1 = node.widgets.find(w => w.name === "audio1"); + const wS1 = node.widgets.find(w => w.name === "start1"); + const wE1 = node.widgets.find(w => w.name === "end1"); + const wA2 = node.widgets.find(w => w.name === "audio2"); + const wS2 = node.widgets.find(w => w.name === "start2"); + const wE2 = node.widgets.find(w => w.name === "end2"); + const wGap = node.widgets.find(w => w.name === "gap_frames"); + const wFps = node.widgets.find(w => w.name === "fps"); + const wTot = node.widgets.find(w => w.name === "total_display"); + + // Nodes 2.0：显示型字段走 options.read_only（Vue 部件没有 element 可改） + if (wTot && wTot.options) wTot.options.read_only = true; + if (wTot) setTimeout(() => { const el = wTot.inputEl || wTot.element; if (el) { el.readOnly = true; el.style.cssText = "background-color:#1a1a1a;color:#00E676;text-align:center;font-weight:bold;font-size:14px;min-width:120px;white-space:nowrap;overflow:hidden;"; } }, 100); + + const audioEl1 = document.createElement("audio"); audioEl1.controls = true; audioEl1.style.cssText = "width:100%;height:30px;outline:none;"; + const audioEl2 = document.createElement("audio"); audioEl2.controls = true; audioEl2.style.cssText = "width:100%;height:30px;outline:none;"; + let dur1 = 0, dur2 = 0, pausing = false, _last1 = null, _last2 = null; + let peaks1 = null, peaks2 = null; + let drags = { 1: null, 2: null }; // 存储双轨的拖动状态 + + const ctr = document.createElement("div"); + Object.assign(ctr.style, { display:"flex", flexDirection:"column", gap:"4px", width:"100%", padding:"6px", boxSizing:"border-box", background:"#161616", borderRadius:"6px", color:"#ccc", fontFamily:"sans-serif", marginTop:"4px" }); + + const mkUpload = (w, label) => { + const fi = document.createElement("input"); fi.type = "file"; fi.accept = "audio/*,video/*"; fi.style.display = "none"; + fi.onchange = async () => { + if (!fi.files.length) return; + const body = new FormData(); body.append("image", fi.files[0]); + const resp = await api.fetchApi("/upload/image", { method: "POST", body }); + if (resp.status === 200) { const d = await resp.json(); const n = d.name || fi.files[0].name; if (!w.options.values.includes(n)) w.options.values.push(n); w.value = n; if (w.callback) w.callback(n); } + }; + document.body.appendChild(fi); + const btn = node.addWidget("button", label, "image", () => { app.canvas.node_widget = null; fi.click(); }); + if (btn.options) btn.options.serialize = false; + return fi; + }; + mkUpload(wA1, "📁 上传音频 1"); mkUpload(wA2, "📁 上传音频 2"); + + const dpr = window.devicePixelRatio || 1; + const label1 = document.createElement("div"); label1.textContent = "🎵 音频1"; label1.style.cssText = "font-size:11px;color:#4FC3F7;font-weight:bold;"; + ctr.appendChild(label1); ctr.appendChild(audioEl1); + const canvas1 = document.createElement("canvas"); canvas1.style.cssText = "width:100%;height:36px;border-radius:3px;background:#111;cursor:pointer;display:block;"; + canvas1.width = 1200 * dpr; canvas1.height = 72 * dpr; ctr.appendChild(canvas1); + + const label2 = document.createElement("div"); label2.textContent = "🎵 音频2"; label2.style.cssText = "font-size:11px;color:#F44336;font-weight:bold;"; + ctr.appendChild(label2); ctr.appendChild(audioEl2); + const canvas2 = document.createElement("canvas"); canvas2.style.cssText = "width:100%;height:36px;border-radius:3px;background:#111;cursor:pointer;display:block;"; + canvas2.width = 1200 * dpr; canvas2.height = 72 * dpr; ctr.appendChild(canvas2); + + const domWidget = node.addDOMWidget("xb3_ui", "custom", ctr); + // Nodes 2.0：DOM widget 高度走 computeLayoutSize/getMinHeight（经典模式仍用 computeSize） + sizeDomWidget(node, domWidget, 240); + if (node.size[1] < 520) node.size[1] = 520; + + const getD = (t) => t === 1 ? (dur1 || 10) : (dur2 || 10); + const getS = (t) => t === 1 ? (parseFloat(wS1?.value) || 0) : (parseFloat(wS2?.value) || 0); + const getE = (t) => { const e = t === 1 ? (parseFloat(wE1?.value) || 0) : (parseFloat(wE2?.value) || 0); return e > 0 ? e : getD(t); }; + const mouseToX = (cx, cvs) => { const r = cvs.getBoundingClientRect(); return (cx - r.left) * (cvs.width / r.width); }; + + const syncW = (t) => { + const s = getS(t), e = getE(t), ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; + const dur = getD(t); if (e > dur) { if (we) we.value = dur; } + if (ws) { ws.value = s; if (ws.inputEl) ws.inputEl.value = s; if (ws.element) ws.element.value = s; } + if (we) { we.value = e; if (we.inputEl) we.inputEl.value = e; if (we.element) we.element.value = e; } + const durSec = Math.max(0, e - s); + const lbl = t === 1 ? label1 : label2; + lbl.textContent = (t === 1 ? "🎵 音频1" : "🎵 音频2") + " (已选择 " + durSec.toFixed(2) + "秒)"; + if (wTot) { + const fps = parseFloat(wFps?.value) || 25, g = parseInt(wGap?.value) || 0; + const d1 = Math.max(0, getE(1) - getS(1)), d2 = Math.max(0, getE(2) - getS(2)); + const totalFrames = Math.max(1, Math.floor(((Math.round(d1 * fps) + g + Math.round(d2 * fps)) + 2) / 4) * 4 + 1); + const tot = totalFrames + " 帧 (" + (d1 + g / fps + d2).toFixed(2) + "s)"; + if (wTot.value !== tot) { wTot.value = tot; if (wTot.inputEl) wTot.inputEl.value = tot; if (wTot.element) wTot.element.value = tot; } + } + }; + + const fetchPeaks = async (filename, track) => { + if (!filename || filename === "none") { if (track === 1) peaks1 = null; else peaks2 = null; return; } + try { + const resp = await api.fetchApi("/xb_toolbox/audio_waveform", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ filename, num_peaks: 4000 }) }); + if (resp.ok) { const data = await resp.json(); if (track === 1) { peaks1 = data.peaks; dur1 = data.duration; } else { peaks2 = data.peaks; dur2 = data.duration; } } + } catch (e) {} + }; + + const visMapper = (cvs, t) => { + const s = getS(t), e = getE(t), dur = getD(t), selW = (e - s) || 0.01; + const sVis = Math.max(0, s - 0.125 * selW), eVis = Math.min(dur, e + 0.125 * selW); + const visW = (eVis - sVis) || 0.01, wwCSS = (cvs.width / dpr) - PAD * 2; + return { sVis, eVis, visW, wwCSS, toCSS: (time) => PAD + ((time - sVis) / visW) * wwCSS, toTime: (cssX) => sVis + ((cssX - PAD) / wwCSS) * visW }; + }; + + const drawWave = (cvs, t) => { + const ctx = cvs.getContext("2d"), W = cvs.width / dpr, H = cvs.height / dpr; + ctx.setTransform(dpr, 0, 0, dpr, 0, 0); + ctx.clearRect(0, 0, W, H); ctx.fillStyle = "#111"; ctx.fillRect(0, 0, W, H); + const d = getD(t), s = getS(t), e = getE(t), vm = visMapper(cvs, t), ww = vm.wwCSS, mid = H / 2; + const peaks = t === 1 ? peaks1 : peaks2; + + if (peaks && peaks.length > 1) { + const startIdx = Math.max(0, Math.floor((vm.sVis / d) * peaks.length)), endIdx = Math.min(peaks.length - 1, Math.ceil((vm.eVis / d) * peaks.length)); + const vp = peaks.slice(startIdx, endIdx + 1); + if (vp.length > 1) { + const DISPLAY_POINTS = Math.min(1000, Math.floor((vm.wwCSS || ww) * 1.5)); + let maxPoints = [], minPoints = []; + const step = vp.length / DISPLAY_POINTS; + for (let i = 0; i < DISPLAY_POINTS; i++) { + const idx = i * step, lo = Math.floor(idx), hi = Math.min(lo + 1, vp.length - 1), frac = idx - lo; + maxPoints.push(vp[lo][0] * (1 - frac) + vp[hi][0] * frac); minPoints.push(vp[lo][1] * (1 - frac) + vp[hi][1] * frac); + } + const absMax = Math.max(Math.max(...maxPoints, 0.001), Math.abs(Math.min(...minPoints, -0.001))); + const scale = (v) => (v / absMax) * (H * 0.48); + + const pathActive = new Path2D(), pathInactive = new Path2D(); + pathActive.moveTo(PAD, mid); pathInactive.moveTo(PAD, mid); + for (let i = 0; i < DISPLAY_POINTS; i++) { + const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; + if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(maxPoints[i])); else pathInactive.lineTo(x, mid - scale(maxPoints[i])); + } + for (let i = DISPLAY_POINTS - 1; i >= 0; i--) { + const x = PAD + (i / (DISPLAY_POINTS - 1)) * ww, tAt = vm.sVis + (i / (DISPLAY_POINTS - 1)) * vm.visW; + if (tAt >= s && tAt <= e) pathActive.lineTo(x, mid - scale(minPoints[i])); else pathInactive.lineTo(x, mid - scale(minPoints[i])); + } + pathActive.closePath(); pathInactive.closePath(); + ctx.fillStyle = "rgba(100,100,100,0.3)"; ctx.fill(pathInactive); ctx.fillStyle = t === 1 ? "#4FC3F7" : "#F44336"; ctx.fill(pathActive); + } + } + + // 处理拖拽边框 + let gx = vm.toCSS(s), rx = vm.toCSS(e); + if (drags[t] && drags[t].type === "start_line" && typeof drags[t].tempX === "number") gx = drags[t].tempX; + if (drags[t] && drags[t].type === "end_line" && typeof drags[t].tempX === "number") rx = drags[t].tempX; + if (gx > rx) { const tmp = gx; gx = rx; rx = tmp; } + + ctx.fillStyle = "rgba(0,0,0,0.65)"; ctx.fillRect(PAD, 0, gx - PAD, H); ctx.fillRect(rx, 0, W - PAD - rx, H); + ctx.fillStyle = "#4CAF50"; ctx.fillRect(gx - 2, 0, 4, H); + ctx.fillStyle = "#F44336"; ctx.fillRect(rx - 2, 0, 4, H); + + const ael = t === 1 ? audioEl1 : audioEl2; + if (ael.readyState >= 1 && ael.currentTime > 0) { + const px = vm.toCSS(ael.currentTime); + if (px > PAD && px < W - PAD) { ctx.fillStyle = "#FFF"; ctx.fillRect(px - 1, 0, 2, H); } + } + + ctx.fillStyle = "#fff"; ctx.font = "bold 10px sans-serif"; + ctx.fillText(s.toFixed(2) + "s", Math.max(PAD + 4, gx - 45), 14); + const ewt = ctx.measureText(e.toFixed(2) + "s").width; + ctx.fillText(e.toFixed(2) + "s", Math.min(W - PAD - ewt - 4, rx + 6), 14); + ctx.setTransform(1, 0, 0, 1, 0, 0); + }; + + // ============================================================ + // 鼠标事件引擎 + // ============================================================ + const bindMouse = (cvs, t) => { + cvs.onmousedown = (ev) => { + if (getD(t) <= 0) return; + const mx = mouseToX(ev.clientX, cvs) / dpr, vm = visMapper(cvs, t); + const sx = vm.toCSS(getS(t)), ex = vm.toCSS(getE(t)); + drags[t] = null; + if (Math.abs(mx - sx) < HANDLE_HIT) { drags[t] = { type: "start_line" }; return; } + if (Math.abs(mx - ex) < HANDLE_HIT) { drags[t] = { type: "end_line" }; return; } + + const clickTime = vm.toTime(mx), ael = t === 1 ? audioEl1 : audioEl2; + if (ael.readyState >= 1) ael.currentTime = Math.max(0, Math.min(getD(t), clickTime)); + }; + + window.addEventListener("mousemove", (ev) => { + const mx = mouseToX(ev.clientX, cvs) / dpr; + if (drags[t]) { + drags[t].tempX = Math.max(PAD, Math.min(cvs.width / dpr - PAD, mx)); + } else { + const rect = cvs.getBoundingClientRect(); + if (ev.clientX >= rect.left && ev.clientX <= rect.right && ev.clientY >= rect.top && ev.clientY <= rect.bottom) { + const vm = visMapper(cvs, t), sx = vm.toCSS(getS(t)), ex = vm.toCSS(getE(t)); + cvs.style.cursor = (Math.abs(mx - sx) < HANDLE_HIT || Math.abs(mx - ex) < HANDLE_HIT) ? "ew-resize" : "pointer"; + } + } + }); + }; + bindMouse(canvas1, 1); bindMouse(canvas2, 2); + + window.addEventListener("mouseup", () => { + [1, 2].forEach(t => { + if (drags[t] && typeof drags[t].tempX === "number") { + const cvs = t === 1 ? canvas1 : canvas2, vm = visMapper(cvs, t), newVal = vm.toTime(drags[t].tempX); + const fps = parseFloat(wFps?.value) || 25, snap = v => Math.round(v * fps) / fps; + const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; + if (drags[t].type === "start_line") ws.value = snap(Math.max(0, Math.min(newVal, getE(t) - 0.1))); + if (drags[t].type === "end_line") we.value = snap(Math.min(getD(t), Math.max(newVal, getS(t) + 0.1))); + syncW(t); const ael = t === 1 ? audioEl1 : audioEl2; if (ael.readyState >= 1) ael.currentTime = getS(t); + } + drags[t] = null; + }); + }); + + // ============================================================ + // 音频逻辑 + // ============================================================ + const updateSrc = (ael, w, t, _last) => { const f = w?.value; if (f && f !== "none" && f !== _last) { if (t === 1) _last1 = f; else _last2 = f; ael.src = api.apiURL("/view?" + new URLSearchParams({ filename: f, type: "input", t: Date.now() })); } }; + const bindAudio = (ael, t) => { + ael.addEventListener("loadedmetadata", () => { if (t === 1) dur1 = ael.duration; else dur2 = ael.duration; const d = getD(t); const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(ws?.value) || 0; const curEnd = parseFloat(we?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > d + 1 || Math.abs(curEnd - 10.0) < 0.001)) { ws.value = 0; we.value = Math.floor(d * fps) / fps; } syncW(t); ael.currentTime = parseFloat(ws?.value) || 0; }); + ael.addEventListener("play", () => { const s = getS(t); if (ael.currentTime < s || ael.currentTime >= getE(t)) ael.currentTime = s; }); + ael.addEventListener("timeupdate", () => { if (pausing) return; if (ael.currentTime >= getE(t)) { pausing = true; ael.pause(); ael.currentTime = getS(t); setTimeout(() => { pausing = false; }, 100); } }); + }; + bindAudio(audioEl1, 1); bindAudio(audioEl2, 2); + + const onTimeChange = (t, resetPlayback) => { syncW(t); if (resetPlayback) { const ael = t === 1 ? audioEl1 : audioEl2; if (ael.readyState >= 1) ael.currentTime = getS(t); } }; + + [wA1, wA2].forEach((w, i) => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); const t = i + 1; const fn = w.value; fetchPeaks(fn, t).then(() => { const ws = t === 1 ? wS1 : wS2, we = t === 1 ? wE1 : wE2; const d = t === 1 ? dur1 : dur2; const fps = parseFloat(wFps?.value) || 25; const curStart = parseFloat(ws?.value) || 0; const curEnd = parseFloat(we?.value) || 0; if (curStart === 0 && (curEnd <= 0 || curEnd > d + 1 || Math.abs(curEnd - 10.0) < 0.001)) { ws.value = 0; we.value = Math.floor(d * fps) / fps; } syncW(t); }); updateSrc(i === 0 ? audioEl1 : audioEl2, w, t, i === 0 ? _last1 : _last2); }; } }); + [wS1].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(1, true); }; } }); + [wE1].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(1, false); }; } }); + [wS2].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(2, true); }; } }); + [wE2].forEach(w => { if (w) { const o = w.callback; w.callback = function () { o?.apply(this, arguments); onTimeChange(2, false); }; } }); + if (wGap) { const o = wGap.callback; wGap.callback = function () { o?.apply(this, arguments); syncW(1); syncW(2); }; } + + let _running = true; + const renderLoop = () => { if (!_running) return; if (dur1 > 0 || dur2 > 0) { syncW(1); syncW(2); drawWave(canvas1, 1); drawWave(canvas2, 2); } requestAnimationFrame(renderLoop); }; + renderLoop(); + + setTimeout(() => { updateSrc(audioEl1, wA1, 1, _last1); updateSrc(audioEl2, wA2, 2, _last2); if (wA1?.value && wA1.value !== "none") fetchPeaks(wA1.value, 1); if (wA2?.value && wA2.value !== "none") fetchPeaks(wA2.value, 2); }, 400); + + const oo = node.onRemoved; node.onRemoved = () => { _running = false; oo?.apply(node); }; + }; + } }); \ No newline at end of file diff --git a/js/xb_audio_slicer_v3.js b/web/xb_audio_slicer_v3.js similarity index 100% rename from js/xb_audio_slicer_v3.js rename to web/xb_audio_slicer_v3.js diff --git a/js/xb_batch_images.js b/web/xb_batch_images.js similarity index 100% rename from js/xb_batch_images.js rename to web/xb_batch_images.js diff --git a/js/xb_cleanup_migrate.js b/web/xb_cleanup_migrate.js similarity index 100% rename from js/xb_cleanup_migrate.js rename to web/xb_cleanup_migrate.js diff --git a/js/xb_comic.js b/web/xb_comic.js similarity index 100% rename from js/xb_comic.js rename to web/xb_comic.js diff --git a/js/xb_compat.js b/web/xb_compat.js similarity index 100% rename from js/xb_compat.js rename to web/xb_compat.js diff --git a/js/xb_cosyvoice3.js b/web/xb_cosyvoice3.js similarity index 100% rename from js/xb_cosyvoice3.js rename to web/xb_cosyvoice3.js diff --git a/js/xb_digital_human_dual.js b/web/xb_digital_human_dual.js similarity index 100% rename from js/xb_digital_human_dual.js rename to web/xb_digital_human_dual.js diff --git a/js/xb_digital_human_single.js b/web/xb_digital_human_single.js similarity index 100% rename from js/xb_digital_human_single.js rename to web/xb_digital_human_single.js diff --git a/js/xb_fast_toggle.js b/web/xb_fast_toggle.js similarity index 100% rename from js/xb_fast_toggle.js rename to web/xb_fast_toggle.js diff --git a/js/xb_flux_encode.js b/web/xb_flux_encode.js similarity index 100% rename from js/xb_flux_encode.js rename to web/xb_flux_encode.js diff --git a/js/xb_folder_picker.js b/web/xb_folder_picker.js similarity index 97% rename from js/xb_folder_picker.js rename to web/xb_folder_picker.js index 370e4b8..72f4f02 100644 --- a/js/xb_folder_picker.js +++ b/web/xb_folder_picker.js @@ -1,34 +1,34 @@ -import { app } from "../../scripts/app.js"; -import { api } from "../../scripts/api.js"; - -// ============================================================ -// XB_BatchFolderLoader — 文件夹选择器 UI -// ============================================================ - -const isZH = navigator.language.startsWith("zh"); - -app.registerExtension({ - name: "XB_ToolBox.FolderPicker", - async beforeRegisterNodeDef(nodeType, nodeData, app) { - if (nodeData.name === "XB_BatchFolderLoader" || nodeData.name === "XB_BatchFolderBatchLoader") { - const onNodeCreated = nodeType.prototype.onNodeCreated; - - nodeType.prototype.onNodeCreated = function () { - const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined; - const dirWidget = this.widgets.find(w => w.name === "directory"); - - const btnLabel = isZH ? "📁 浏览文件夹..." : "📁 Browse Folder..."; - const btnWidget = this.addWidget("button", btnLabel, "browse", () => { - api.fetchApi("/xb_toolbox/choose_folder", { method: "POST" }) - .then(res => res.json()) - .then(data => { - if (data.path) { - dirWidget.value = data.path; - } - }).catch(err => {}); - }); - return r; - }; - } - } +import { app } from "../../scripts/app.js"; +import { api } from "../../scripts/api.js"; + +// ============================================================ +// XB_BatchFolderLoader — 文件夹选择器 UI +// ============================================================ + +const isZH = navigator.language.startsWith("zh"); + +app.registerExtension({ + name: "XB_ToolBox.FolderPicker", + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (nodeData.name === "XB_BatchFolderLoader" || nodeData.name === "XB_BatchFolderBatchLoader") { + const onNodeCreated = nodeType.prototype.onNodeCreated; + + nodeType.prototype.onNodeCreated = function () { + const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined; + const dirWidget = this.widgets.find(w => w.name === "directory"); + + const btnLabel = isZH ? "📁 浏览文件夹..." : "📁 Browse Folder..."; + const btnWidget = this.addWidget("button", btnLabel, "browse", () => { + api.fetchApi("/xb_toolbox/choose_folder", { method: "POST" }) + .then(res => res.json()) + .then(data => { + if (data.path) { + dirWidget.value = data.path; + } + }).catch(err => {}); + }); + return r; + }; + } + } }); \ No newline at end of file diff --git a/js/xb_gallery_picker.js b/web/xb_gallery_picker.js similarity index 100% rename from js/xb_gallery_picker.js rename to web/xb_gallery_picker.js diff --git a/js/xb_hailuo_video.js b/web/xb_hailuo_video.js similarity index 100% rename from js/xb_hailuo_video.js rename to web/xb_hailuo_video.js diff --git a/js/xb_image_prompt_preset.js b/web/xb_image_prompt_preset.js similarity index 100% rename from js/xb_image_prompt_preset.js rename to web/xb_image_prompt_preset.js diff --git a/js/xb_image_prompt_preset_pro.js b/web/xb_image_prompt_preset_pro.js similarity index 100% rename from js/xb_image_prompt_preset_pro.js rename to web/xb_image_prompt_preset_pro.js diff --git a/js/xb_image_scale.js b/web/xb_image_scale.js similarity index 100% rename from js/xb_image_scale.js rename to web/xb_image_scale.js diff --git a/js/xb_label.js b/web/xb_label.js similarity index 100% rename from js/xb_label.js rename to web/xb_label.js diff --git a/js/xb_list_dispatcher.js b/web/xb_list_dispatcher.js similarity index 100% rename from js/xb_list_dispatcher.js rename to web/xb_list_dispatcher.js diff --git a/js/xb_llama_model_loader_pro.js b/web/xb_llama_model_loader_pro.js similarity index 100% rename from js/xb_llama_model_loader_pro.js rename to web/xb_llama_model_loader_pro.js diff --git a/js/xb_llama_prompt_reverse.js b/web/xb_llama_prompt_reverse.js similarity index 100% rename from js/xb_llama_prompt_reverse.js rename to web/xb_llama_prompt_reverse.js diff --git a/js/xb_load_video.js b/web/xb_load_video.js similarity index 100% rename from js/xb_load_video.js rename to web/xb_load_video.js diff --git a/js/xb_master_param.js b/web/xb_master_param.js similarity index 97% rename from js/xb_master_param.js rename to web/xb_master_param.js index 9c44fef..996e23c 100644 --- a/js/xb_master_param.js +++ b/web/xb_master_param.js @@ -1,79 +1,79 @@ -import { app } from "../../scripts/app.js"; - -// ============================================================ -// XB_MasterParameter — 万能参数控制器 UI -// ============================================================ - -const MODE_CONFIGS = { - "Free Mode": { min: 0, max: 9999, dragStep: 10, snapRound: 1, precision: 0, def: 0 }, - "Model Mode": { min: 0, max: 50, dragStep: 10, snapRound: 1, precision: 0, def: 0 }, - "Encode Mode": { min: 64, max: 3840, dragStep: 320, snapRound: 32, precision: 0, def: 64 }, - "Decode Mode": { min: 64, max: 3840, dragStep: 320, snapRound: 32, precision: 0, def: 64 }, - "Ratio Mode": { min: 0.00, max: 1.00, dragStep: 0.1, snapRound: 0.01, precision: 2, def: 0.50 }, - "Other Mode": { min: -99999.00, max: 99999.00, dragStep: 0.1, snapRound: 0.01, precision: 2, def: 1.00 } -}; - -app.registerExtension({ - name: "xiaobai.masterparams", - async nodeCreated(node) { - if (node.comfyClass === "XB_MasterParameter") { - const wMode = node.widgets.find(w => w.name === "mode"); - const wValueOrig = node.widgets.find(w => w.name === "value"); - - if (wMode && wValueOrig) { - const valueWidgetIndex = node.widgets.indexOf(wValueOrig); - - node._xb_shadow_widgets = {}; - for (const [mName, cfg] of Object.entries(MODE_CONFIGS)) { - const w = node.addWidget("number", "value", cfg.def, function(v){}, { - min: cfg.min, - max: cfg.max, - step: cfg.dragStep, - round: cfg.snapRound, - precision: cfg.precision - }); - node._xb_shadow_widgets[mName] = node.widgets.pop(); - } - - node.swapWidget = function(modeName) { - const currIdx = node.widgets.findIndex(w => w.name === "value"); - if (currIdx > -1) { - node.widgets.splice(currIdx, 1); - } - - const targetW = node._xb_shadow_widgets[modeName]; - if (targetW) { - targetW.name = "value"; - node.widgets.splice(valueWidgetIndex, 0, targetW); - } - app.graph.setDirtyCanvas(true, true); - }; - - node.swapWidget(wMode.value); - - const origModeCb = wMode.callback; - wMode.callback = function(v) { - node.swapWidget(v); - if (origModeCb) return origModeCb.apply(this, arguments); - }; - - const onConfigure = node.onConfigure; - node.onConfigure = function(info) { - if (onConfigure) onConfigure.apply(this, arguments); - - if (wMode.value) { - node.swapWidget(wMode.value); - - if (info && info.widgets_values && info.widgets_values.length > valueWidgetIndex) { - const savedVal = info.widgets_values[valueWidgetIndex]; - const activeW = node.widgets.find(w => w.name === "value"); - if (activeW && savedVal !== undefined) { - activeW.value = savedVal; - } - } - } - }; - } - } - } +import { app } from "../../scripts/app.js"; + +// ============================================================ +// XB_MasterParameter — 万能参数控制器 UI +// ============================================================ + +const MODE_CONFIGS = { + "Free Mode": { min: 0, max: 9999, dragStep: 10, snapRound: 1, precision: 0, def: 0 }, + "Model Mode": { min: 0, max: 50, dragStep: 10, snapRound: 1, precision: 0, def: 0 }, + "Encode Mode": { min: 64, max: 3840, dragStep: 320, snapRound: 32, precision: 0, def: 64 }, + "Decode Mode": { min: 64, max: 3840, dragStep: 320, snapRound: 32, precision: 0, def: 64 }, + "Ratio Mode": { min: 0.00, max: 1.00, dragStep: 0.1, snapRound: 0.01, precision: 2, def: 0.50 }, + "Other Mode": { min: -99999.00, max: 99999.00, dragStep: 0.1, snapRound: 0.01, precision: 2, def: 1.00 } +}; + +app.registerExtension({ + name: "xiaobai.masterparams", + async nodeCreated(node) { + if (node.comfyClass === "XB_MasterParameter") { + const wMode = node.widgets.find(w => w.name === "mode"); + const wValueOrig = node.widgets.find(w => w.name === "value"); + + if (wMode && wValueOrig) { + const valueWidgetIndex = node.widgets.indexOf(wValueOrig); + + node._xb_shadow_widgets = {}; + for (const [mName, cfg] of Object.entries(MODE_CONFIGS)) { + const w = node.addWidget("number", "value", cfg.def, function(v){}, { + min: cfg.min, + max: cfg.max, + step: cfg.dragStep, + round: cfg.snapRound, + precision: cfg.precision + }); + node._xb_shadow_widgets[mName] = node.widgets.pop(); + } + + node.swapWidget = function(modeName) { + const currIdx = node.widgets.findIndex(w => w.name === "value"); + if (currIdx > -1) { + node.widgets.splice(currIdx, 1); + } + + const targetW = node._xb_shadow_widgets[modeName]; + if (targetW) { + targetW.name = "value"; + node.widgets.splice(valueWidgetIndex, 0, targetW); + } + app.graph.setDirtyCanvas(true, true); + }; + + node.swapWidget(wMode.value); + + const origModeCb = wMode.callback; + wMode.callback = function(v) { + node.swapWidget(v); + if (origModeCb) return origModeCb.apply(this, arguments); + }; + + const onConfigure = node.onConfigure; + node.onConfigure = function(info) { + if (onConfigure) onConfigure.apply(this, arguments); + + if (wMode.value) { + node.swapWidget(wMode.value); + + if (info && info.widgets_values && info.widgets_values.length > valueWidgetIndex) { + const savedVal = info.widgets_values[valueWidgetIndex]; + const activeW = node.widgets.find(w => w.name === "value"); + if (activeW && savedVal !== undefined) { + activeW.value = savedVal; + } + } + } + }; + } + } + } }); \ No newline at end of file diff --git a/js/xb_memory_viz.css b/web/xb_memory_viz.css similarity index 100% rename from js/xb_memory_viz.css rename to web/xb_memory_viz.css diff --git a/js/xb_memory_viz.js b/web/xb_memory_viz.js similarity index 100% rename from js/xb_memory_viz.js rename to web/xb_memory_viz.js diff --git a/js/xb_memory_viz_graph.js b/web/xb_memory_viz_graph.js similarity index 100% rename from js/xb_memory_viz_graph.js rename to web/xb_memory_viz_graph.js diff --git a/js/xb_minimax_hailuo_link.js b/web/xb_minimax_hailuo_link.js similarity index 100% rename from js/xb_minimax_hailuo_link.js rename to web/xb_minimax_hailuo_link.js diff --git a/js/xb_model_loader_v1_gguf.js b/web/xb_model_loader_v1_gguf.js similarity index 100% rename from js/xb_model_loader_v1_gguf.js rename to web/xb_model_loader_v1_gguf.js diff --git a/js/xb_model_loader_v2_gguf.js b/web/xb_model_loader_v2_gguf.js similarity index 100% rename from js/xb_model_loader_v2_gguf.js rename to web/xb_model_loader_v2_gguf.js diff --git a/js/xb_model_loader_v3_gguf.js b/web/xb_model_loader_v3_gguf.js similarity index 100% rename from js/xb_model_loader_v3_gguf.js rename to web/xb_model_loader_v3_gguf.js diff --git a/js/xb_msr.js b/web/xb_msr.js similarity index 100% rename from js/xb_msr.js rename to web/xb_msr.js diff --git a/js/xb_node_runtime.js b/web/xb_node_runtime.js similarity index 100% rename from js/xb_node_runtime.js rename to web/xb_node_runtime.js diff --git a/js/xb_node_sizer.js b/web/xb_node_sizer.js similarity index 100% rename from js/xb_node_sizer.js rename to web/xb_node_sizer.js diff --git a/js/xb_play_sound.js b/web/xb_play_sound.js similarity index 100% rename from js/xb_play_sound.js rename to web/xb_play_sound.js diff --git a/js/xb_qwen_encode.js b/web/xb_qwen_encode.js similarity index 100% rename from js/xb_qwen_encode.js rename to web/xb_qwen_encode.js diff --git a/js/xb_qwen_prompt_preset.js b/web/xb_qwen_prompt_preset.js similarity index 100% rename from js/xb_qwen_prompt_preset.js rename to web/xb_qwen_prompt_preset.js diff --git a/js/xb_reference_any.js b/web/xb_reference_any.js similarity index 100% rename from js/xb_reference_any.js rename to web/xb_reference_any.js diff --git a/js/xb_relay_ui.js b/web/xb_relay_ui.js similarity index 97% rename from js/xb_relay_ui.js rename to web/xb_relay_ui.js index 9fed01e..fe79e0f 100644 --- a/js/xb_relay_ui.js +++ b/web/xb_relay_ui.js @@ -1,94 +1,94 @@ -import { app } from "../../scripts/app.js"; -import { installRelayLock, setWidgetValue } from "./xb_compat.js"; - -// ============================================================ -// XB_Wan_RelayNode — 首尾帧接力点 UI 交互 -// ============================================================ -// 【Nodes 2.0 适配说明】 -// · 原来的 🔒 蒙版画在 onDrawForeground、点击拦截靠 onMouseDown —— 2.0 下这两个钩子都不会被调用， -// 现统一改为：端口被连线接管时 **直接隐藏本地部件** + DOM widget 提示条（经典/2.0 通用）。 -// · 原来在 onDrawBackground 里把 this.imgs 置空以屏蔽预览的写法 2.0 下无效， -// 改为在锁定状态变化时整体收走 / 归还 node.imgs（不再依赖绘制钩子）。 -// · 经典模式下 onConnectionsChange 仍会触发，因此两种模式行为一致。 - -app.registerExtension({ - name: "xiaobai.relay_node_ui_master", - async beforeRegisterNodeDef(nodeType, nodeData, app) { - if (nodeData.name !== "XB_Wan_RelayNode") return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - installRelayLock(node, { - text: "🔒 尾图已由连线接管", - bannerName: "xb_relay_banner", - names: ["end_image_file", "image_upload", "upload"], - hideImgs: true, - locked: (n) => { - const inp = n.inputs?.find((i) => i.name === "opt_end_image"); - return inp?.link != null; - }, - }); - }; - }, -}); - -// ============================================================ -// 🧮 _New 接力点自动计算：总计 = 单次帧数 × 接力数量 − 重叠帧数 × (接力数量−1) -// 重叠参数已在接力节点上，直接读本地 widget，无需轮询 -// ============================================================ -const RELAY_NODES_CALC = [ - "XB_WanAnimate_RelayNode_New", - "XB_WanInfiniteTalk_RelayNode_New", - "XB_WanSCAIL_RelayNode_New", - "XB_WanInfiniteTalk_RelayNode_MultiRef", - "XB_WanInfiniteTalk_RelayNode_AllInOne", -]; - -const RELAY_OVERLAP = { - "XB_WanAnimate_RelayNode_New": "continue_motion_max_frames", - "XB_WanInfiniteTalk_RelayNode_New": "motion_frame_count", - "XB_WanSCAIL_RelayNode_New": "previous_frame_count", - "XB_WanInfiniteTalk_RelayNode_MultiRef": "motion_frame_count", - "XB_WanInfiniteTalk_RelayNode_AllInOne": "motion_frame_count", -}; - -app.registerExtension({ - name: "xiaobai.relay_new_auto_calc", - async beforeRegisterNodeDef(nodeType, nodeData, app) { - if (!RELAY_NODES_CALC.includes(nodeData.name)) return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - const wSeg = node.widgets?.find(w => w.name === "segment_length"); - const wCnt = node.widgets?.find(w => w.name === "relay_count"); - const wDisp = node.widgets?.find(w => w.name === "total_frames_display"); - const ovName = RELAY_OVERLAP[node.comfyClass]; - const wOvl = ovName ? node.widgets?.find(w => w.name === ovName) : null; - - if (!wSeg || !wCnt || !wDisp) return; - - const update = () => { - const seg = parseInt(wSeg.value) || 0; - const cnt = parseInt(wCnt.value) || 0; - if (seg <= 0 || cnt <= 0) { setWidgetValue(wDisp, ""); return; } - const ovl = wOvl ? (parseInt(wOvl.value) || 0) : 0; - setWidgetValue(wDisp, String(Math.max(1, seg * cnt - ovl * (cnt - 1)))); - app.graph.setDirtyCanvas(true, false); - }; - - [wSeg, wCnt, wOvl].forEach(w => { - if (!w) return; - const orig = w.callback; - w.callback = function(v) { if (orig) orig.call(this, v); update(); }; - }); - - setTimeout(update, 50); - }; - } -}); +import { app } from "../../scripts/app.js"; +import { installRelayLock, setWidgetValue } from "./xb_compat.js"; + +// ============================================================ +// XB_Wan_RelayNode — 首尾帧接力点 UI 交互 +// ============================================================ +// 【Nodes 2.0 适配说明】 +// · 原来的 🔒 蒙版画在 onDrawForeground、点击拦截靠 onMouseDown —— 2.0 下这两个钩子都不会被调用， +// 现统一改为：端口被连线接管时 **直接隐藏本地部件** + DOM widget 提示条（经典/2.0 通用）。 +// · 原来在 onDrawBackground 里把 this.imgs 置空以屏蔽预览的写法 2.0 下无效， +// 改为在锁定状态变化时整体收走 / 归还 node.imgs（不再依赖绘制钩子）。 +// · 经典模式下 onConnectionsChange 仍会触发，因此两种模式行为一致。 + +app.registerExtension({ + name: "xiaobai.relay_node_ui_master", + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (nodeData.name !== "XB_Wan_RelayNode") return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + installRelayLock(node, { + text: "🔒 尾图已由连线接管", + bannerName: "xb_relay_banner", + names: ["end_image_file", "image_upload", "upload"], + hideImgs: true, + locked: (n) => { + const inp = n.inputs?.find((i) => i.name === "opt_end_image"); + return inp?.link != null; + }, + }); + }; + }, +}); + +// ============================================================ +// 🧮 _New 接力点自动计算：总计 = 单次帧数 × 接力数量 − 重叠帧数 × (接力数量−1) +// 重叠参数已在接力节点上，直接读本地 widget，无需轮询 +// ============================================================ +const RELAY_NODES_CALC = [ + "XB_WanAnimate_RelayNode_New", + "XB_WanInfiniteTalk_RelayNode_New", + "XB_WanSCAIL_RelayNode_New", + "XB_WanInfiniteTalk_RelayNode_MultiRef", + "XB_WanInfiniteTalk_RelayNode_AllInOne", +]; + +const RELAY_OVERLAP = { + "XB_WanAnimate_RelayNode_New": "continue_motion_max_frames", + "XB_WanInfiniteTalk_RelayNode_New": "motion_frame_count", + "XB_WanSCAIL_RelayNode_New": "previous_frame_count", + "XB_WanInfiniteTalk_RelayNode_MultiRef": "motion_frame_count", + "XB_WanInfiniteTalk_RelayNode_AllInOne": "motion_frame_count", +}; + +app.registerExtension({ + name: "xiaobai.relay_new_auto_calc", + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (!RELAY_NODES_CALC.includes(nodeData.name)) return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + const wSeg = node.widgets?.find(w => w.name === "segment_length"); + const wCnt = node.widgets?.find(w => w.name === "relay_count"); + const wDisp = node.widgets?.find(w => w.name === "total_frames_display"); + const ovName = RELAY_OVERLAP[node.comfyClass]; + const wOvl = ovName ? node.widgets?.find(w => w.name === ovName) : null; + + if (!wSeg || !wCnt || !wDisp) return; + + const update = () => { + const seg = parseInt(wSeg.value) || 0; + const cnt = parseInt(wCnt.value) || 0; + if (seg <= 0 || cnt <= 0) { setWidgetValue(wDisp, ""); return; } + const ovl = wOvl ? (parseInt(wOvl.value) || 0) : 0; + setWidgetValue(wDisp, String(Math.max(1, seg * cnt - ovl * (cnt - 1)))); + app.graph.setDirtyCanvas(true, false); + }; + + [wSeg, wCnt, wOvl].forEach(w => { + if (!w) return; + const orig = w.callback; + w.callback = function(v) { if (orig) orig.call(this, v); update(); }; + }); + + setTimeout(update, 50); + }; + } +}); diff --git a/js/xb_role_scene_dispatcher.js b/web/xb_role_scene_dispatcher.js similarity index 100% rename from js/xb_role_scene_dispatcher.js rename to web/xb_role_scene_dispatcher.js diff --git a/js/xb_save_video.js b/web/xb_save_video.js similarity index 100% rename from js/xb_save_video.js rename to web/xb_save_video.js diff --git a/js/xb_shenbi.js b/web/xb_shenbi.js similarity index 100% rename from js/xb_shenbi.js rename to web/xb_shenbi.js diff --git a/js/xb_smart_ui.js b/web/xb_smart_ui.js similarity index 97% rename from js/xb_smart_ui.js rename to web/xb_smart_ui.js index 63363ef..d52bc44 100644 --- a/js/xb_smart_ui.js +++ b/web/xb_smart_ui.js @@ -1,36 +1,36 @@ -import { app } from "../../scripts/app.js"; -import { installRelayLock } from "./xb_compat.js"; - -// ============================================================ -// XB_Wan_RelayNode — 接力点智能 UI（端口被连线接管时锁定本地部件） -// ============================================================ -// 【Nodes 2.0 适配说明】 -// 旧实现靠 widget.disabled + 改 label + 直接改 inputEl/element 样式，在 Nodes 2.0 下： -// · 数字/下拉是 Vue 组件，没有 inputEl/element（样式与 pointerEvents 失效） -// · widget.disabled 在经典画布下不起作用（部件照样能点） -// 现在统一交给 installRelayLock（与 xb_relay_ui.js 同一套实现，靠 node._xbRelayLock 去重， -// 两个扩展同时存在也不会重复生效）。 - -app.registerExtension({ - name: "XB_ToolBox.SmartUI", - async beforeRegisterNodeDef(nodeType, nodeData, app) { - if (nodeData.name !== "XB_Wan_RelayNode") return; - - const onNodeCreated = nodeType.prototype.onNodeCreated; - nodeType.prototype.onNodeCreated = function () { - if (onNodeCreated) onNodeCreated.apply(this, arguments); - const node = this; - - installRelayLock(node, { - text: "🔒 端口已被连线接管", - bannerName: "xb_relay_banner", - names: ["end_image_file", "image_upload", "upload"], - hideImgs: true, - locked: (n) => { - const inp = n.inputs?.find((i) => i.name === "opt_end_image"); - return inp?.link != null; - }, - }); - }; - }, -}); +import { app } from "../../scripts/app.js"; +import { installRelayLock } from "./xb_compat.js"; + +// ============================================================ +// XB_Wan_RelayNode — 接力点智能 UI（端口被连线接管时锁定本地部件） +// ============================================================ +// 【Nodes 2.0 适配说明】 +// 旧实现靠 widget.disabled + 改 label + 直接改 inputEl/element 样式，在 Nodes 2.0 下： +// · 数字/下拉是 Vue 组件，没有 inputEl/element（样式与 pointerEvents 失效） +// · widget.disabled 在经典画布下不起作用（部件照样能点） +// 现在统一交给 installRelayLock（与 xb_relay_ui.js 同一套实现，靠 node._xbRelayLock 去重， +// 两个扩展同时存在也不会重复生效）。 + +app.registerExtension({ + name: "XB_ToolBox.SmartUI", + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (nodeData.name !== "XB_Wan_RelayNode") return; + + const onNodeCreated = nodeType.prototype.onNodeCreated; + nodeType.prototype.onNodeCreated = function () { + if (onNodeCreated) onNodeCreated.apply(this, arguments); + const node = this; + + installRelayLock(node, { + text: "🔒 端口已被连线接管", + bannerName: "xb_relay_banner", + names: ["end_image_file", "image_upload", "upload"], + hideImgs: true, + locked: (n) => { + const inp = n.inputs?.find((i) => i.name === "opt_end_image"); + return inp?.link != null; + }, + }); + }; + }, +}); diff --git a/js/xb_status_switch.js b/web/xb_status_switch.js similarity index 100% rename from js/xb_status_switch.js rename to web/xb_status_switch.js diff --git a/js/xb_storyboard_processor.js b/web/xb_storyboard_processor.js similarity index 100% rename from js/xb_storyboard_processor.js rename to web/xb_storyboard_processor.js diff --git a/js/xb_string_merge.js b/web/xb_string_merge.js similarity index 100% rename from js/xb_string_merge.js rename to web/xb_string_merge.js diff --git a/js/xb_video.js b/web/xb_video.js similarity index 100% rename from js/xb_video.js rename to web/xb_video.js diff --git a/js/xb_video_loader.js b/web/xb_video_loader.js similarity index 100% rename from js/xb_video_loader.js rename to web/xb_video_loader.js diff --git a/js/xb_video_preview.js b/web/xb_video_preview.js similarity index 100% rename from js/xb_video_preview.js rename to web/xb_video_preview.js diff --git a/js/xb_wan_dancer_combo.js b/web/xb_wan_dancer_combo.js similarity index 100% rename from js/xb_wan_dancer_combo.js rename to web/xb_wan_dancer_combo.js diff --git a/js/xb_yue2_lyrics.js b/web/xb_yue2_lyrics.js similarity index 100% rename from js/xb_yue2_lyrics.js rename to web/xb_yue2_lyrics.js diff --git a/js/xb_yue2_score.js b/web/xb_yue2_score.js similarity index 100% rename from js/xb_yue2_score.js rename to web/xb_yue2_score.js diff --git a/js/xb_yue2_style_prompt.js b/web/xb_yue2_style_prompt.js similarity index 100% rename from js/xb_yue2_style_prompt.js rename to web/xb_yue2_style_prompt.js