diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d024f29..3348710 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -142,9 +142,9 @@ covered by the `styles.css` manifest. voice-group shape (`node_value` = what the node's voice widget takes), so cards, sidebar and Script Director share one implementation. A missing engine stays in the switch marked "not installed" with install steps; nothing else depends on it. - GPT-SoVITS data comes only from the node's HTTP contract, version 4 + GPT-SoVITS data comes only from the node's HTTP contract, version 9 (`/anomalous_tts/characters`, `/audio`, `/settings`, `/status`, the storage, - library, pretrained, browse and import routes; the node repo's `docs/INTERFACE.md`, + library, pretrained, browse, import and import preview routes; the node repo's `docs/INTERFACE.md`, mirrored as the project doc `anomalous-tts-interface.md`); Anomalous never reads or writes its model folders. `loadGptSovitsStatus` caches the setup status with the engine data (null for a node without `/status`, which hides setup and import). The character list is a @@ -174,26 +174,42 @@ covered by the `styles.css` manifest. drop area then one card per draft, one unfolded at a time, `add` = files for an existing character, also opened straight from a card's "Add files"), and runs a spotlight tour once per screen ("?" replays it). It owns the drafts (characters being built; `target` = files added to an - existing one) and the unassigned tray, every file row (in exactly one draft's - `rows`), the uploads (three at a time), a debounced inspect per draft, and imports - the ready drafts one after another (each commit on its own, without the rows - `leftOut` names; closing discards uncommitted uploads). `draw()` rebuilds the cards on structural changes, `refresh()` only - repaints status, so typing never loses focus. - `tts_import_groups.js` holds the pure rules: which draft a file goes to (weight - stems; other files follow the weights in their nearest folder, then folder names - or file-name prefixes), which files are left out (`leftOut`: clips outside 3–10 s - and their line files), which folders are never taken (GPT-SoVITS program and - training folders, the same list as the node's scan), where a draft stands, and its - checklist (the next step first). `ui_tts_import_sources.js` brings files in: the - browser's dialogs and drops (uploaded), the node's picker (paths), leaving out - those folders and offering the picker before a very large upload. `ui_tts_import_draft.js` draws a draft's card + existing one), the unassigned tray, the characters waiting for the next batch + (a large add opens one batch; the next opens on request or once the batch is + imported), every file row (in exactly one draft's `rows`; a row keeps its own + name in `original` and gets a clash-free `name` in a draft, sent to the node as + the file's `name`), a debounced inspect per draft, and imports the ready drafts + one after another (each commit on its own, without the rows `leftOut` names; + closing discards uncommitted uploads). `draw()` rebuilds the cards on structural + changes, `refresh()` only repaints status, so typing never loses focus. + `ui_tts_import_uploads.js` sends browser files in chunks, three at a time, once + they are in a draft (tray files wait); `ui_tts_import_player.js` plays one clip + at a time for comparing (browser files from memory, local paths through the + node's preview route). + `tts_import_groups.js` holds the pure rules: which folders are never taken + (`skipFolder` / `isPackage`, the same rules as the node's scan: Python + environments and base models anywhere, a package's program and training folders + only inside a package, never the chosen folder), which draft a file goes to + (`groupFiles`: a folder with one character's weights and clips is that character; + other weights by stem; other files follow the character folder, the weights in + their nearest folder, then folder names or file-name prefixes; unclaimed `.list` + files go to every character), how an add splits into groups and batches + (`planGroups`, `splitBatch`), new names for files that would clash inside a + character (`clashFreeNames`), which files are left out (`leftOut`: clips outside + 3–10 s and their line files), where a draft stands, and its checklist (the next + step first). `ui_tts_import_sources.js` brings files in: the browser's dialogs and + drops (uploaded), the node's picker (paths); every source gives folders starting + with the chosen one, says which folders were left out or too deep, and offers the + picker before a very large upload. `ui_tts_import_draft.js` draws a draft's card (name in the header, steps left, the checklist with the next step's button and the chosen weights, one line per clip with its owner picker, the first clips with "show all", left-out files folded, line files, language) and the unassigned card - (one folded line per folder when large); `ui_tts_import_screens.js` holds the fixed - screens (the first question, the batch drop area) and the tour steps, and builds each row's elements once, so they - survive redraws and moves. `ui_tts_file_drop.js` reads OS - drops (walking dropped folders) for the studio, the sidebar and the workbench. `ui_tts_path_picker.js` picks server-side folders or files + (one folded line per folder when large), and builds each row's elements once, so + they survive redraws and moves; `ui_tts_import_screens.js` holds the fixed + screens (the first question, the batch drop area, the "add more" menu) and the + tour steps. `ui_tts_file_drop.js` reads OS drops (walking dropped folders, only + files an import can use, up to 5000) for the studio, the sidebar and the + workbench. `ui_tts_path_picker.js` picks server-side folders or files through the node's browse route, since the browser cannot see local paths. - `audio_node_targets.js` is the single table of canvas nodes the audio studio writes into and what each takes: F5-TTS (`F5TTSAudio`, `F5TTSAudioAdvanced`, diff --git a/CHANGELOG.md b/CHANGELOG.md index fad786b..47cdad9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ - **Usage guide (使用说明)**: in the audio page, the header **!** opens a five-step how-to; the full guide is `docs/guides/audio-studio.md`. - **Engine switch: F5-TTS / GPT-SoVITS (引擎切换)**: the audio page now manages voices per engine. Installed engines are detected at runtime; a missing one stays visible as "not installed" with install steps instead of disappearing, and nothing else depends on it. GPT-SoVITS characters come from the separate Anomalous_TTS node pack: preview reference clips, pick a reference clip per emotion ("Edit emotions", saved through the node's API), drag a character onto the node, and push scripts with the Script Director. - **Audio sidebar (音频侧栏)**: characters are grouped by language (GPT-SoVITS) or folder (F5-TTS) in foldable groups; a character unfolds into its clips, and a click plays one. Characters drag onto nodes from the sidebar too, a red dot marks characters that need a look, a search box appears from six characters on, and files dropped on a GPT-SoVITS character open **Add files** for it. -- **GPT-SoVITS setup and import (GPT-SoVITS 准备与导入)**: an empty studio is a drop area with **Import character**, which also sits in the toolbar. **GPT-SoVITS settings** at the bottom of the sidebar opens a dialog with where characters are kept, pretrained files and missing Python packages; missing pretrained files only get a small dot on that entry, with "don't remind me", once your characters need them, and the import form notes what the chosen language will download. **Batch import (批量导入)**: the import window takes one or many characters. It first asks what you have (one character, many characters or a package, or files for a character you already have), with three steps at the top and a spotlight tour the first time ("?" replays it). Each character card leads with a checklist that lights up the next step (GPT model, SoVITS model, clips, main voice, name) and offers its button right there; the card says how many steps are left. Drop files or whole folders, or pick a folder on this PC (a GPT-SoVITS package): weights pair up by name (latest epoch kept), clips and line files follow the weights in their folder (a folder per character works even when the names differ) or the character their folder or file name points at, and anything unclear waits in a card of its own, one line per folder that can be assigned at once. Clips outside 3–10 s are left out with their line files, program and training folders (`runtime`, `pretrained_models`, `logs`…) are skipped, long clip lists show the first 60, and a very large upload first suggests picking the folder on this PC instead; uploads run three at a time and a big batch no longer makes the window sluggish. Each character is a card: one line with its name, a short status and what it holds, unfolding into its model tiles and clips; files move between characters with the picker on each row. **Import N characters** imports each ready one on its own, so one failure leaves the others alone. Status is colour-coded throughout: green ready, amber still missing something, red a failed or refused file, blue uploading; **Import** stays at the bottom. Adding to a character skips files it already has and merges annotation files (Anomalous_TTS interface 5). There is one storage place (default `models/gpt_sovits`); changing it can move the existing characters there with progress. Download pretrained files with progress or use an existing GPT-SoVITS package, all without editing `extra_model_paths.yaml` or restarting. **Import character** builds a character from dropped or picked GPT / SoVITS weights, reference clips and annotation files: the name, SoVITS version, clip lengths and lines are filled in automatically, and everything is copied (originals stay). Drop files on a card, or use **Add files**, to add clips and emotions to an existing character. Lines are found in same-name `.txt` / `.lab` files, annotation files or, as a marked guess, the file name; a text file dropped on a clip's line box fills it. Needs Anomalous_TTS with interface version 4; file actions only work on the computer running ComfyUI. +- **GPT-SoVITS setup and import (GPT-SoVITS 准备与导入)**: an empty studio is a drop area with **Import character**, which also sits in the toolbar. **GPT-SoVITS settings** at the bottom of the sidebar opens a dialog with where characters are kept, pretrained files and missing Python packages; missing pretrained files only get a small dot on that entry, with "don't remind me", once your characters need them, and the import form notes what the chosen language will download. **Batch import (批量导入)**: the import window takes one or many characters. It first asks what you have (one character, many characters or a package, or files for a character you already have), with three steps at the top and a spotlight tour the first time ("?" replays it). Each character card leads with a checklist that lights up the next step (GPT model, SoVITS model, clips, main voice, name) and offers its button right there; the card says how many steps are left. Drop files or whole folders, or pick a folder on this PC (a GPT-SoVITS package): a folder holding one set of GPT + SoVITS weights and its clips is one character named after the folder (two characters whose weights are both called `model` stay apart; `派蒙/v2` becomes "派蒙 v2"); weights in a package's weight folders pair up by name (latest epoch kept); clips and line files follow their character folder, the weights in their folder or the character their folder or file name points at; an annotation file (`.list`) no character claims goes to every character; two clips with the same name (`开心/01.wav`, `难过/01.wav`) both go in, the second as `难过_01.wav`; anything unclear waits in a card of its own, one line per folder that can be assigned at once, and is uploaded only once it is put into a character. Clips outside 3–10 s are left out with their line files. Python environments and base models (`runtime`, `pretrained_models`, `venv`…) are skipped, and a package's program and training folders (`GPT_SoVITS`, `logs`, `output`, `TEMP`, `tools`) only inside a package: your own folders called `output` or `GPT_SoVITS`, and the folder you chose or dropped, are always read; the window says which folders were skipped or too deep (6 levels) instead of files going missing quietly. A big add opens 20 characters (about 800 files) at a time; **Next batch** opens the rest, and it opens by itself once a batch is imported, so a huge folder does not freeze the window. Every clip has **▶** to listen while importing — uploading, unassigned or picked on this PC — and clicking another clip switches to it for quick comparing. Long clip lists show the first 60, and a very large upload first suggests picking the folder on this PC instead; uploads run three at a time. Each character is a card: one line with its name, a short status and what it holds, unfolding into its model tiles and clips; files move between characters with the picker on each row. **Import N characters** imports each ready one on its own, so one failure leaves the others alone. Status is colour-coded throughout: green ready, amber still missing something, red a failed or refused file, blue uploading; **Import** stays at the bottom. Adding to a character skips files it already has and merges annotation files (Anomalous_TTS interface 5). There is one storage place (default `models/gpt_sovits`); changing it can move the existing characters there with progress. Download pretrained files with progress or use an existing GPT-SoVITS package, all without editing `extra_model_paths.yaml` or restarting. **Import character** builds a character from dropped or picked GPT / SoVITS weights, reference clips and annotation files: the name, SoVITS version, clip lengths and lines are filled in automatically, and everything is copied (originals stay). Drop files on a card, or use **Add files**, to add clips and emotions to an existing character. Lines are found in same-name `.txt` / `.lab` files, annotation files or, as a marked guess, the file name; a text file dropped on a clip's line box fills it. Needs Anomalous_TTS with interface version 9; file actions only work on the computer running ComfyUI. ### Fixes - All-in-one loader recipes and cross-node prompt injection use one verified adapter table; prompt text never crosses between positive and negative. diff --git a/docs/guides/audio-studio.md b/docs/guides/audio-studio.md index 67acc43..ca18e13 100644 --- a/docs/guides/audio-studio.md +++ b/docs/guides/audio-studio.md @@ -158,15 +158,19 @@ Load Audio 正好相反：它只用一条具体的音频，所以要拖某一行 做完一项就打勾；卡片标题旁写着“还差 N 步”，全部完成变成绿色的 **可以导入**。任何时候都可以直接把文件拖进窗口，会自动放到对的地方。 -参考音频一段一行：填上情绪名（例如“开心”）就成了 `{开心}`。**不在 3~10 秒的音频不会导入**（节点用不了它们），连同同名的台词文件一起收进卡片底部的“不导入”，点开能看到是哪些。一个角色音频很多时先显示前 60 段，点 **显示全部** 看其余的。台词会自动从同名 `.txt` / `.lab` 或标注文件里找；台词框是黄框表示台词取自文件名，请确认；红色虚线框表示没找到，可以粘贴，也可以把 `.txt` / `.lab` / `.list` 拖到框里。卡片底部是台词文件和 **角色说的语言**（一般自动判断就行）。 +参考音频一段一行：点前面的 **▶** 试听，再点一次（■）停止；点另一段就直接换过去，方便一段段对比着挑。上传中、还没分给角色的、从这台电脑选的音频都能听。填上情绪名（例如“开心”）就成了 `{开心}`。**不在 3~10 秒的音频不会导入**（节点用不了它们），连同同名的台词文件一起收进卡片底部的“不导入”，点开能看到是哪些。一个角色音频很多时先显示前 60 段，点 **显示全部** 看其余的。台词会自动从同名 `.txt` / `.lab` 或标注文件里找；台词框是黄框表示台词取自文件名，请确认；红色虚线框表示没找到，可以粘贴，也可以把 `.txt` / `.lab` / `.list` 拖到框里。卡片底部是台词文件和 **角色说的语言**（一般自动判断就行）。 **很多角色 / 整合包**时： -- 模型按名字配成角色：`ALuoNa-e15.ckpt` 和 `ALuoNa_e16_s224.pth` 都算 `ALuoNa`。好几轮（e5、e10、e15）时自动用轮数最大的。 -- 一个角色一个文件夹时（比如 `模型/阿罗娜/` 里有 `成品模型/` 和 `参考音频/`），文件夹里的音频和台词都归这个文件夹里的模型，名字对不上也没关系。同一个文件夹里有 `Guang` 和 `Guang_v4` 两套模型时，归 `Guang`。 -- 其他的按所在的文件夹名（比如 `参考音频/ALuoNa/`）或文件名开头分给对应的角色。 -- 分不出来的放在最上面一张黄色卡片里。文件多时一个文件夹一行，可以把整个文件夹一次放到某个角色；点开这一行，也能一个一个选。分错的文件，右边也能改。 -- GPT-SoVITS 程序和训练用的文件夹（`runtime`、`pretrained_models`、`logs`、`output`、`TEMP`、`tools` 等）会跳过：里面是底模、训练中间存档和数据集切片，不是角色文件。 +- **一个文件夹一个角色**：一个文件夹里（包括它的子文件夹）只有一套 GPT + SoVITS 模型，还有音频或台词，它就是一个角色，用文件夹名当角色名。比如 `派蒙/` 里有 `model-e10.ckpt`、`model_e10.pth` 和参考音频，角色就叫“派蒙”；两个角色的模型都叫 `model` 也不会混在一起。文件夹叫 `v2`、`v4` 这类版本名时，前面加上上一级的名字（“派蒙 v2”）。 +- 模型放在整合包的 `GPT_weights*` / `SoVITS_weights*` 里，或者好几个角色的模型平铺在一起时，按模型文件名配成角色：`ALuoNa-e15.ckpt` 和 `ALuoNa_e16_s224.pth` 都算 `ALuoNa`。好几轮（e5、e10、e15）时自动用轮数最大的。 +- 其他的按所在的文件夹（同一个文件夹里的模型）、文件夹名（比如 `参考音频/ALuoNa/`）或文件名开头分给对应的角色。同一个文件夹里有 `Guang` 和 `Guang_v4` 两套模型时，归 `Guang`。 +- 没分给任何角色的 `.list` 标注文件，每个角色都放一份（节点按文件名找台词，不会找错）。 +- 同一个角色里有同名的文件（比如 `开心/01.wav` 和 `难过/01.wav`），后来的会在前面加上文件夹名（`难过_01.wav`），同名的台词文件一起改，不会丢；鼠标停在文件名上能看到原来的文件。 +- 分不出来的放在最上面一张黄色卡片里，放进角色后才上传。文件多时一个文件夹一行，可以把整个文件夹一次放到某个角色；点开这一行，也能一个一个选。分错的文件，右边也能改。 +- 不会读的文件夹：`runtime`、`pretrained_models`、`venv` 这类 Python 环境和底模文件夹；整合包里（旁边有 `runtime`、`GPT_weights*`、`webui.py` 这些）的 `GPT_SoVITS`、`logs`、`output`、`TEMP`、`tools`，里面是程序、训练中间存档和数据集切片。你自己起名叫 `output`、`GPT_SoVITS` 的文件夹照常读；你直接选中或拖进来的那个文件夹一定会读。跳过了哪些，窗口里会写出来。 +- 从这台电脑选文件夹时往下读 6 层，更深的文件夹会写出来，需要的话单独选它再加一次。一次最多读 5000 个文件。 +- 角色很多时先打开一批（20 个角色，或者大约 800 个文件），其余的在“打开下一批”按钮里；这一批导入完，下一批会自动打开。这样一次拖进很大的文件夹也不会卡。 - 一次要上传很多文件（超过 1500 个或 2 GB）时会先问你：ComfyUI 就在这台电脑上，改用 **从这台电脑选文件夹** 直接复制，不用上传，快得多，界面也不会卡。 - 卡片收起时只有一行，点开看详情，一次只展开一张。右上角 **+ 继续添加** 可以再加文件，**+ 新角色** 手动加一个角色。 @@ -224,4 +228,5 @@ Load Audio 正好相反：它只用一条具体的音频，所以要拖某一行 | GPT-SoVITS 角色卡片上提示“没有找到合适的主声音” | 角色文件夹里没有 3~10 秒、找得到台词的音频。点“编辑情绪”手动选一条 | | 新放进去的 GPT-SoVITS 角色或音频没出现 | 点 **刷新音色**。节点那边在 ComfyUI 里按 **R** 刷新下拉框 | | 侧栏里看不到“GPT-SoVITS 设置”，工具栏上没有“导入角色” | Anomalous_TTS 版本太旧（没有接口版本 3）。更新 Anomalous_TTS 后重启 ComfyUI | +| 导入时从这台电脑选的音频放不出来，或者同名文件导入时报“有重名的文件” | Anomalous_TTS 早于接口版本 9。更新后重启 ComfyUI | | “导入角色”是灰的 | 页面是从别的设备打开的。导入会写文件，只能在运行 ComfyUI 的电脑上做 | diff --git a/web/modules/locales.js b/web/modules/locales.js index caa5076..3bb1e80 100644 --- a/web/modules/locales.js +++ b/web/modules/locales.js @@ -1552,7 +1552,8 @@ export const i18n = { ttsUploadLarge: "这次要上传 {count} 个文件，共 {size}。浏览器上传会很慢，也会让界面变卡。ComfyUI 就在这台电脑上，改用“从这台电脑选文件夹”可以直接复制，不用上传。", ttsUploadUseLocal: "从这台电脑选文件夹", ttsUploadAnyway: "仍然上传", - ttsSkippedFolders: "跳过了 GPT-SoVITS 程序和训练用的文件夹：{folders}。", + ttsSkippedFolders: "跳过了这些文件夹（Python 环境、底模，或整合包的程序和训练文件夹）：{folders}。", + ttsScanTooDeep: "这些文件夹太深，没有列出：{folders}。需要的话直接选那个文件夹再加一次。", ttsImportKeptBestMany: "每个角色的 GPT 和 SoVITS 都用了轮数最大的那个（另外 {count} 个没用）。", ttsImportAddTitle: "添加文件 · {name}", ttsImportNameTaken: "已经有叫“{name}”的角色了。", @@ -1572,12 +1573,22 @@ export const i18n = { ttsImportPickLocal: "从本机路径选择…", ttsImportPickLocalHint: "直接选择电脑上的文件（比如整合包里的 GPT_weights_v2 / SoVITS_weights_v2），不用上传，更快。", ttsImportSkipped: "跳过了不支持或重复的文件：{files}", + ttsImportRenamed: "有 {count} 个文件和同一角色里的文件重名，已在前面加上所在文件夹的名字。", + ttsImportRenamedFrom: "改了名，原文件：{file}", + ttsBatchLater: "文件较多，先打开了一批；还有 {count} 个角色在下一批，导入完这批会自动打开。", + ttsBatchNext: "打开下一批（{count} 个角色）", + ttsBatchImported: "上一批的 {count} 个角色已导入，打开了下一批。", + ttsFooterPending: "下一批还有 {count} 个角色", + ttsPreviewStop: "停止试听", + ttsPreviewFailed: "放不了：{error}", + ttsPreviewUnreadable: "浏览器读不了这个音频", ttsImportRowSkip: "角色里已有，跳过", ttsImportRowMerge: "合并新台词", ttsImportRowConflict: "角色里有同名的不同文件", ttsImportRowUnsupported: "不支持", ttsImportExistingDifferent: "角色里已经有同名但内容不同的文件：{file}。改个文件名再放进来，或者先删掉这一行。", ttsImportRowUploading: "上传中 {percent}%", + ttsImportRowNotSent: "放进角色后上传", ttsImportRowReady: "✓ 就绪", ttsImportRowError: "✕ {error}", ttsImportSeconds: "{seconds} 秒", @@ -3286,7 +3297,8 @@ export const i18n = { ttsUploadLarge: "This would upload {count} files, {size} in all. Uploading through the browser is slow and makes the window sluggish. ComfyUI runs on this computer: choosing the folder from this computer copies it directly, without uploading.", ttsUploadUseLocal: "Choose the folder on this computer", ttsUploadAnyway: "Upload anyway", - ttsSkippedFolders: "Skipped GPT-SoVITS program and training folders: {folders}.", + ttsSkippedFolders: "Skipped these folders (Python environments, base models, or a package's program and training folders): {folders}.", + ttsScanTooDeep: "These folders are too deep and were not listed: {folders}. Choose such a folder directly to add it.", ttsImportKeptBestMany: "Each character uses its GPT and SoVITS weights of the latest epoch ({count} others not used).", ttsImportAddTitle: "Add files · {name}", ttsImportNameTaken: "There is already a character named \"{name}\".", @@ -3306,12 +3318,22 @@ export const i18n = { ttsImportPickLocal: "Choose from this computer…", ttsImportPickLocalHint: "Pick files on the computer directly (e.g. a package's GPT_weights_v2 / SoVITS_weights_v2): no upload, faster.", ttsImportSkipped: "Skipped unsupported or duplicate files: {files}", + ttsImportRenamed: "{count} files had the same name as another file of the character; their folder's name was put in front.", + ttsImportRenamedFrom: "Renamed; original file: {file}", + ttsBatchLater: "Many files: one batch is open; {count} more characters are in the next batch, which opens once this one is imported.", + ttsBatchNext: "Next batch ({count} characters)", + ttsBatchImported: "The {count} characters of the last batch are imported; here is the next batch.", + ttsFooterPending: "{count} characters in the next batch", + ttsPreviewStop: "Stop", + ttsPreviewFailed: "Cannot play: {error}", + ttsPreviewUnreadable: "the browser cannot read this audio", ttsImportRowSkip: "already there, skipped", ttsImportRowMerge: "new lines merged", ttsImportRowConflict: "a different file has this name", ttsImportRowUnsupported: "not supported", ttsImportExistingDifferent: "The character already has a different file named {file}. Rename it and add it again, or remove that row.", ttsImportRowUploading: "uploading {percent}%", + ttsImportRowNotSent: "uploads once in a character", ttsImportRowReady: "✓ ready", ttsImportRowError: "✕ {error}", ttsImportSeconds: "{seconds} s", diff --git a/web/modules/tts_import_groups.js b/web/modules/tts_import_groups.js index 5af40c5..53d4469 100644 --- a/web/modules/tts_import_groups.js +++ b/web/modules/tts_import_groups.js @@ -2,31 +2,81 @@ import { importProblem, rowState, usableAsReference } from './tts_setup_api.js'; /** * Batch import: sort dropped or scanned files into the characters being built. - * Pure rules, no DOM. A GPT / SoVITS weight names its character (the file name - * without the epoch part); clips and line files follow the folder they are in or - * the start of their name, else the only character there is, else they wait - * unassigned for the user. + * Pure rules, no DOM. A folder holding one character's GPT and SoVITS weights is + * that character, named by the folder; other weights name their character by the + * file name without the epoch part. Clips and line files follow the character + * folder they are in, then the folder their weights are in, the start of their + * name, else the only character there is, else they wait unassigned for the user. */ const WEIGHT_TAIL = { gpt: /-e\d+.*$/i, sovits: /_e\d+.*$/i }; const SIDECAR = /\.(txt|lab)$/i; +const ANNOTATION = /\.list$/i; +const WEIGHT_DIR = /^(gpt|sovits)_weights/i; +const VERSION_DIR = /^v\d/i; // `v2`, `v2Pro`: named with the folder above (`派蒙 v2`) /** - * Folders of the GPT-SoVITS program and its training runs (base models, training - * checkpoints, dataset slices, tool models): never a character's files. The same - * list as the node's folder scan (`SCAN_SKIP` in Anomalous_TTS `core/browse.py`). + * Folders never walked: Python environments and base models, anywhere; the program + * and training folders of a GPT-SoVITS package (checkpoints, dataset slices, tool + * models) only inside a package, since users name their own folders `output` or + * `GPT_SoVITS` too. The same rules as the node's folder scan (Anomalous_TTS + * `core/browse.py`, `skip_folder` / `is_package`). */ -const SKIP_FOLDERS = new Set(['runtime', 'gpt_sovits', 'pretrained_models', 'logs', 'output', 'temp', 'tools', - '__pycache__', 'site-packages', 'venv', 'node_modules']); +const ALWAYS_SKIP = new Set(['runtime', 'pretrained_models', '__pycache__', 'site-packages', 'venv', 'node_modules']); +const PACKAGE_SKIP = new Set(['gpt_sovits', 'logs', 'output', 'temp', 'tools']); +const PACKAGE_FILES = new Set(['webui.py', 'api.py', 'api_v2.py', 'inference_webui.py', 's1_train.py', 's2_train.py']); +export const DROP_FILE_LIMIT = 5000; // files taken from one drop or folder, like the node's scan -export function isSkippedFolder(name) { - const lower = String(name || '').toLowerCase(); - return SKIP_FOLDERS.has(lower) || lower.startsWith('.'); +/** Whether a folder holding `names` (its sub-folders and files) is a GPT-SoVITS package or program. */ +export function isPackage(names) { + return [...names].some((name) => { + const lower = String(name).toLowerCase(); + return lower === 'runtime' || PACKAGE_FILES.has(lower) || WEIGHT_DIR.test(lower); + }); } -/** Whether a file whose folder is `dir` (`a/b`, relative to what was dropped) sits in a skipped folder. */ -export function inSkippedFolder(dir) { - return String(dir || '').split('/').some(isSkippedFolder); +/** Whether sub-folder `name` of a folder is left out (`inPackage`: that folder is a package); hidden ones always are. */ +export function skipFolder(name, inPackage) { + const lower = String(name || '').toLowerCase(); + return lower.startsWith('.') || ALWAYS_SKIP.has(lower) || (inPackage && PACKAGE_SKIP.has(lower)); +} + +/** + * For a flat file list (the browser's folder dialog): `{ kept, skipped }`, the indexes of + * files outside left-out folders and those folders. `files`: `{ dir, name }`, `dir` starting + * with the chosen folder, which is never left out. + */ +export function sortOutFolders(files) { + const children = new Map(); // folder -> names of its files and sub-folders + const addChild = (dir, name) => { + if (!children.has(dir)) children.set(dir, new Set()); + children.get(dir).add(name); + }; + for (const file of files) { + const parts = String(file.dir || '').split('/').filter(Boolean); + parts.forEach((part, i) => addChild(parts.slice(0, i).join('/'), part)); + addChild(parts.join('/'), file.name); + } + const verdict = new Map(); // dir -> the left-out folder above it, or '' + const skippedOf = (dir) => { + if (verdict.has(dir)) return verdict.get(dir); + const parts = dir.split('/').filter(Boolean); + let found = ''; + for (let i = 1; i < parts.length && !found; i++) { + const parent = parts.slice(0, i).join('/'); + if (skipFolder(parts[i], isPackage(children.get(parent) || []))) found = parts.slice(0, i + 1).join('/'); + } + verdict.set(dir, found); + return found; + }; + const kept = []; + const skipped = new Set(); + files.forEach((file, i) => { + const folder = skippedOf(String(file.dir || '')); + if (folder) skipped.add(folder); + else kept.push(i); + }); + return { kept, skipped: [...skipped] }; } /** @@ -66,6 +116,10 @@ export function weightStem(name, kind) { return base.replace(WEIGHT_TAIL[kind], '').replace(/^[\s_-]+|[\s_-]+$/g, '') || base; } +const parentOf = dir => (dir.includes('/') ? dir.slice(0, dir.lastIndexOf('/')) : ''); +const baseOf = dir => dir.slice(dir.lastIndexOf('/') + 1); +const within = (dir, folder) => dir === folder || dir.startsWith(`${folder}/`); + /** Longest draft key a folder on `dir` or the file name points at, or null. */ function matchKey(item, keys) { const segments = String(item.dir || '').split('/').map(groupKey).filter(segment => segment.length >= 2); @@ -85,63 +139,191 @@ function matchKey(item, keys) { function ownerByFolder(dir, weights) { let at = String(dir || ''); while (at) { - const inside = new Set(weights.filter(w => w.dir === at || w.dir.startsWith(`${at}/`)).map(w => w.key)); + const inside = new Set(weights.filter(w => within(w.dir, at)).map(w => w.key)); if (inside.size) { const list = [...inside]; return list.find(key => list.every(other => other.startsWith(key))) ?? null; } - at = at.includes('/') ? at.slice(0, at.lastIndexOf('/')) : ''; + at = parentOf(at); } return null; } -/** The top folder of `dir`; a package's own weight folders count as the top of the drop. */ -function topFolder(dir) { - const top = String(dir || '').split('/')[0]; - return /^(gpt|sovits)_weights/i.test(top) ? '' : top; +/** + * Character folders: the deepest folders whose weights (sub-folders included) are GPT + * weights of one name and SoVITS weights of one name, and which hold clips or line + * files too: `派蒙/model-e10.ckpt` + `派蒙/model_e10.pth` + `派蒙/1.wav` → `派蒙` + * (`派蒙/模型/`, weights only, is not). Not the top of what was added when that has no + * name, and not a folder whose weights sit in `GPT_weights*` / `SoVITS_weights*` + * folders (a package: its characters are named by their weights). A version folder + * (`v2`) is named with the folder above it. `weights`: `[{ dir, kind, stem }]`; + * `otherDirs`: the folders of the other files. Returns `Map(folder → { key, name })`, + * keys unique among themselves. + */ +function characterFolders(weights, otherDirs) { + const candidates = new Set(); + for (const w of weights) for (let at = w.dir; at; at = parentOf(at)) candidates.add(at); + const qualifies = [...candidates].filter((folder) => { + const inside = weights.filter(w => within(w.dir, folder)); + const stems = kind => new Set(inside.filter(w => w.kind === kind).map(w => w.stem)); + if (stems('gpt').size !== 1 || stems('sovits').size !== 1) return false; + if (!otherDirs.some(dir => within(dir, folder))) return false; + return !inside.some(w => w.dir.slice(folder.length).split('/').some(part => WEIGHT_DIR.test(part))); + }); + const deepest = qualifies.filter(folder => !qualifies.some(other => other !== folder && within(other, folder))); + const found = new Map(); + const used = new Set(); + for (const folder of deepest.sort()) { + const base = baseOf(folder); + const name = VERSION_DIR.test(base) && parentOf(folder) ? `${baseOf(parentOf(folder))} ${base}` : base; + let key = groupKey(name); + if (!key || used.has(key)) key = groupKey(folder); + used.add(key); + found.set(folder, { key, name }); + } + return found; +} + +/** The segments all `dirs` share at their start: the chosen folder, when one folder was added. */ +function commonRoot(dirs) { + const lists = dirs.map(dir => String(dir || '').split('/').filter(Boolean)); + const root = []; + for (let i = 0; lists.length && lists.every(parts => parts.length > i && parts[i] === lists[0][i]); i++) root.push(lists[0][i]); + return root.length; } /** * `items`: `{ name, dir, kind }`; `drafts`: the characters already open, `{ key, target }`. - * Returns `{ added: [{ key, name }], assign: [key | null per item] }`: new characters to - * open (one per weight stem not open yet, or one blank one when nothing else is open), - * and where each item goes: weights by their name, other files by the folder their - * weights sit in (a folder per character), else by folder or file name. Adding to one - * existing character takes everything into it. + * Returns `{ added: [{ key, name }], assign: [key | null per item], shared: [index] }`: new + * characters to open (character folders and weight stems not open yet, or one blank one + * when nothing else is open), where each item goes (see the module comment), and + * annotation files (`.list`) no character claims: they hold lines for everyone's clips, + * so each character gets a copy. Adding to one existing character takes everything into it. */ export function groupFiles(items, drafts) { const keys = drafts.map(draft => draft.key); const added = []; - const soleTarget = drafts.length === 1 && drafts[0].target ? drafts[0].key : null; - if (!soleTarget) { - for (const item of items) { - if (item.kind !== 'gpt' && item.kind !== 'sovits') continue; - const name = weightStem(item.name, item.kind); - const key = groupKey(name); - if (key && !keys.includes(key)) { - keys.push(key); - added.push({ key, name }); - } + const addKey = (key, name) => { + if (key && !keys.includes(key)) { + keys.push(key); + added.push({ key, name }); } + }; + const soleTarget = drafts.length === 1 && drafts[0].target ? drafts[0].key : null; + const isWeight = item => item.kind === 'gpt' || item.kind === 'sovits'; + const weights = items.map((item, index) => ({ item, index })).filter(({ item }) => isWeight(item)) + .map(({ item, index }) => ({ index, dir: String(item.dir || ''), kind: item.kind, name: weightStem(item.name, item.kind) })) + .map(w => ({ ...w, stem: groupKey(w.name) })) + .filter(w => w.stem); + const otherDirs = items.filter(item => !isWeight(item)).map(item => String(item.dir || '')); + const folders = soleTarget ? new Map() : characterFolders(weights, otherDirs); + const folderOf = (dir) => { + for (let at = String(dir || ''); at; at = parentOf(at)) if (folders.has(at)) return folders.get(at); + return null; + }; + for (const w of weights) { + const folder = folderOf(w.dir); + w.key = folder ? folder.key : w.stem; + if (!soleTarget) addKey(w.key, folder ? folder.name : w.name); } if (!keys.length) { keys.push(''); added.push({ key: '', name: '' }); } const only = keys.length === 1 ? keys[0] : null; - const weights = items.filter(item => item.kind === 'gpt' || item.kind === 'sovits') - .map(item => ({ dir: String(item.dir || ''), key: groupKey(weightStem(item.name, item.kind)) })) - .filter(w => w.key); + const weightKey = new Map(weights.map(w => [w.index, w.key])); // The only character takes the rest, unless its weights sit in their own folder and the // file in another one: then it is someone else's (or nobody's) and waits unassigned. + const depth = commonRoot(items.map(item => item.dir)); + const topFolder = (dir) => { + const top = String(dir || '').split('/').filter(Boolean)[depth] || ''; + return WEIGHT_DIR.test(top) ? '' : top; // a package's own weight folders count as the top + }; const weightTops = new Set(weights.map(w => topFolder(w.dir))); const fallback = item => (weightTops.size && !weightTops.has('') && !weightTops.has(topFolder(item.dir)) ? null : only); - const assign = items.map(item => { + const assign = items.map((item, index) => { if (soleTarget !== null) return soleTarget; - if (item.kind === 'gpt' || item.kind === 'sovits') return groupKey(weightStem(item.name, item.kind)) || only; - return ownerByFolder(item.dir, weights) ?? matchKey(item, keys.filter(Boolean)) ?? fallback(item); + if (isWeight(item)) return weightKey.get(index) ?? only; + return folderOf(item.dir)?.key ?? ownerByFolder(item.dir, weights) ?? matchKey(item, keys.filter(Boolean)) ?? fallback(item); }); - return { added, assign }; + const shared = keys.some(Boolean) ? items.map((item, i) => i).filter(i => assign[i] === null && ANNOTATION.test(items[i].name)) : []; + return { added, assign, shared }; +} + +/** + * One add as groups, one per character it goes to: `{ key, name, open, tray, items }`, + * characters already open first (`open`), then new ones in the order they were found, + * then the files no character claims (`tray`). Shared annotation files go into every + * character group of the add. `drafts` as for groupFiles. + */ +export function planGroups(items, drafts) { + const { added, assign, shared } = groupFiles(items, drafts); + const groups = new Map(drafts.map(draft => [draft.key, { key: draft.key, name: '', open: true, tray: false, items: [] }])); + for (const add of added) groups.set(add.key, { key: add.key, name: add.name, open: false, tray: false, items: [] }); + const tray = { key: null, name: '', open: false, tray: true, items: [] }; + const common = new Set(shared); + items.forEach((item, i) => { + if (!common.has(i)) (assign[i] === null ? tray : groups.get(assign[i])).items.push(item); + }); + const list = [...groups.values()].filter(group => group.items.length); + for (const group of list.length ? list : [tray]) group.items.push(...shared.map(i => items[i])); + return tray.items.length ? [...list, tray] : list; +} + +/** + * New names for files that would clash inside one character: two `01.wav` from the + * folders `开心` and `难过` (GPT-SoVITS keeps clips in one folder). The first folder + * keeps its names; files from another folder get its name in front (`难过_01.wav`), + * clip and same-name line file alike, so they still pair up. `items`: `{ name, dir, kind }` + * to place; `taken`: `{ name, dir }` already there. Weights are never renamed (one per + * character). Returns the name for each item. + */ +export function clashFreeNames(items, taken = []) { + const stem = name => name.replace(/\.[^.]+$/, ''); + const used = new Set(); + const owner = new Map(); // lower-case stem -> the folder whose files use it + const claim = (name, dir) => { + used.add(name.toLowerCase()); + if (!owner.has(stem(name.toLowerCase()))) owner.set(stem(name.toLowerCase()), dir); + }; + for (const row of taken) claim(String(row.name), String(row.dir || '')); + return items.map((item) => { + const name = String(item.name); + const dir = String(item.dir || ''); + if (item.kind === 'gpt' || item.kind === 'sovits') return name; + const lower = name.toLowerCase(); + const holder = owner.get(stem(lower)); + if (!used.has(lower) && (holder === undefined || holder === dir)) { + claim(name, dir); + return name; + } + const prefix = (baseOf(dir) || 'file').replace(/[<>:"/\\|?*]/g, '_'); + let candidate = `${prefix}_${name}`; + for (let n = 2; used.has(candidate.toLowerCase()); n++) candidate = `${prefix}${n}_${name}`; + claim(candidate, dir); + return candidate; + }); +} + +/** + * How much of a large add opens now: groups going to characters already open always, + * then new characters in order until `characters` of them or `files` files (at least + * one), and the unassigned files. The rest waits for "next batch", so a big folder + * does not open hundreds of cards and uploads at once. `groups`: `{ open, tray, items }`. + */ +export function splitBatch(groups, { characters = 20, files = 800 } = {}) { + const now = []; + const later = []; + let count = 0; + let size = 0; + for (const group of groups) { + if (group.open || group.tray) { now.push(group); continue; } + if (later.length || (count && (count >= characters || size + group.items.length > files))) { later.push(group); continue; } + now.push(group); + count++; + size += group.items.length; + } + return { now, later }; } /** diff --git a/web/modules/tts_setup_api.js b/web/modules/tts_setup_api.js index d46b9dd..555ce87 100644 --- a/web/modules/tts_setup_api.js +++ b/web/modules/tts_setup_api.js @@ -1,7 +1,7 @@ import { invalidateEngineCache } from './audio_engines.js'; /** - * GPT-SoVITS setup and import through the Anomalous_TTS node (interface v4, §5.2–5.3). + * GPT-SoVITS setup and import through the Anomalous_TTS node (interface v9, §5.2–5.3). * The node owns every file: the storage place, pretrained files and imported * characters. This module only calls its API and holds the pure rules the import * form needs; it has no DOM. @@ -84,6 +84,11 @@ export function scanFolder(path, signal) { return request(`/anomalous_tts/browse?recursive=1&path=${encodeURIComponent(path)}`, { method: 'GET', signal }); } +/** Where the browser can play a local audio file picked by path (the node serves it). */ +export function previewUrl(path) { + return `/anomalous_tts/import/preview?path=${encodeURIComponent(path)}`; +} + // ---------- import ---------- /** @@ -129,7 +134,7 @@ export function discardUploads(ids) { /** * The commit request for the import form. `rows` are in file order: - * { spec: {upload}|{path}, kind, emotion, text }. `language` '' = let the node decide. + * { spec: {upload}|{path} (plus `name` when renamed), kind, emotion, text }. `language` '' = let the node decide. */ export function buildImportBody({ target = null, library = '', character = '', rows, referenceIndex = null, language = '' }) { const settings = {}; @@ -260,11 +265,13 @@ export function missingForLanguage(status, language) { } /** - * One import row at a glance: `tone` colours it (busy | ready | skip | merge | error) + * One import row at a glance: `tone` colours it (busy | ready | skip | merge | error | idle) * and `key` / `params` are its state text. `existing` comes from inspect with a target. + * `queued` false: a file nobody claims yet, uploaded once it is put into a character. */ -export function rowState({ error = '', uploaded = false, progress = 0, existing = null, unsupported = false }) { +export function rowState({ error = '', uploaded = false, queued = true, progress = 0, existing = null, unsupported = false }) { if (error) return { tone: 'error', key: 'ttsImportRowError', params: { error } }; + if (!uploaded && !queued) return { tone: 'idle', key: 'ttsImportRowNotSent', params: {} }; if (!uploaded) return { tone: 'busy', key: 'ttsImportRowUploading', params: { percent: Math.floor(100 * progress) } }; if (existing === 'different') return { tone: 'error', key: 'ttsImportRowConflict', params: {} }; if (unsupported) return { tone: 'error', key: 'ttsImportRowUnsupported', params: {} }; diff --git a/web/modules/ui_tts_file_drop.js b/web/modules/ui_tts_file_drop.js index 63591db..fa91172 100644 --- a/web/modules/ui_tts_file_drop.js +++ b/web/modules/ui_tts_file_drop.js @@ -1,5 +1,6 @@ import { t } from './interface_settings.js'; -import { isSkippedFolder } from './tts_import_groups.js'; +import { importKind } from './tts_setup_api.js'; +import { DROP_FILE_LIMIT, isPackage, skipFolder } from './tts_import_groups.js'; /** * Files dropped from the OS on the audio studio, the sidebar or the import @@ -7,29 +8,48 @@ import { isSkippedFolder } from './tts_import_groups.js'; * from (dropped folders are walked), so the workbench can sort them. */ -/** GPT-SoVITS program and training folders inside a drop are not walked: `{ skipped: folder }`. */ -async function walkEntry(entry, dir, top = false) { - if (entry.isFile) return [{ file: await new Promise((ok, fail) => entry.file(ok, fail)), dir }]; - const sub = dir ? `${dir}/${entry.name}` : entry.name; - if (!top && isSkippedFolder(entry.name)) return [{ skipped: sub }]; - const reader = entry.createReader(); - const children = []; - for (;;) { - const batch = await new Promise((ok, fail) => reader.readEntries(ok, fail)); - if (!batch.length) break; - children.push(...batch); - } - return (await Promise.all(children.map(child => walkEntry(child, sub)))).flat(); +const readAll = reader => new Promise((ok, fail) => reader.readEntries(ok, fail)); + +/** + * The entries of a drop, walked: `{ file, dir }` for each file an import can use, + * `{ skipped: folder }` for a folder left out (tts_import_groups.js skipFolder: never + * one that was dropped by itself), `{ truncated: true }` once `DROP_FILE_LIMIT` files + * were taken. Other files are not even opened: a program folder holds thousands. + */ +async function walkEntries(entries, dir, inPackage, count) { + const sorted = [...entries].sort((a, b) => a.name.localeCompare(b.name)); + const lists = await Promise.all(sorted.map(async (entry) => { + const at = dir ? `${dir}/${entry.name}` : entry.name; + if (entry.isFile) { + if (!importKind(entry.name)) return []; + if (count.files >= DROP_FILE_LIMIT) { count.truncated = true; return []; } + count.files++; + return [{ file: await new Promise((ok, fail) => entry.file(ok, fail)), dir }]; + } + if (inPackage !== null && skipFolder(entry.name, inPackage)) return [{ skipped: at }]; + const reader = entry.createReader(); + const children = []; + for (;;) { + const batch = await readAll(reader); + if (!batch.length) break; + children.push(...batch); + } + return walkEntries(children, at, isPackage(children.map(child => child.name)), count); + })); + return lists.flat(); // in name order, whatever finished first } /** * Files of a drop, with the folder each came from (dropped folders are walked). * The entries must be taken during the drop event, so call it synchronously there. + * Several things dropped together are judged as one folder's contents. */ export function readDroppedFiles(dataTransfer) { const entries = [...(dataTransfer?.items || [])].map(item => item.webkitGetAsEntry?.()).filter(Boolean); if (!entries.length) return Promise.resolve([...(dataTransfer?.files || [])].map(file => ({ file, dir: '' }))); - return Promise.all(entries.map(entry => walkEntry(entry, '', true))).then(lists => lists.flat()); + const count = { files: 0, truncated: false }; + const together = entries.length > 1 ? isPackage(entries.map(entry => entry.name)) : null; + return walkEntries(entries, '', together, count).then(list => (count.truncated ? [...list, { truncated: true }] : list)); } /** diff --git a/web/modules/ui_tts_import.js b/web/modules/ui_tts_import.js index bb349d7..9cf73f5 100644 --- a/web/modules/ui_tts_import.js +++ b/web/modules/ui_tts_import.js @@ -3,15 +3,17 @@ import { createViewScope } from './ui_lifecycle.js'; import { anomalousAlert, anomalousConfirm } from './ui_dialog.js'; import { loadEngine, loadGptSovitsStatus } from './audio_engines.js'; import { - buildImportBody, commitImport, discardUploads, importKind, inspectImport, nameConflict, pickWeights, uploadFile, + buildImportBody, commitImport, discardUploads, importKind, inspectImport, nameConflict, pickWeights, } from './tts_setup_api.js'; -import { draftChecklist, draftState, groupFiles, groupKey, leftOut } from './tts_import_groups.js'; +import { clashFreeNames, draftChecklist, draftState, groupKey, leftOut, planGroups, splitBatch } from './tts_import_groups.js'; import { TRAY, renderDraftCard, renderTrayCard, showSource, syncReference, updateRow } from './ui_tts_import_draft.js'; import { PICKER_OVERLAY_CLASS } from './ui_tts_path_picker.js'; import { readDroppedFiles } from './ui_tts_file_drop.js'; import { createFileSources } from './ui_tts_import_sources.js'; +import { createImportPlayer } from './ui_tts_import_player.js'; +import { createUploadQueue } from './ui_tts_import_uploads.js'; import { closeSpotlightTour, isSpotlightTourActive, startSpotlightTour } from './ui_spotlight_tour.js'; -import { TOURS, markTourSeen, renderBatchHero, renderChooseScreen, seenTours } from './ui_tts_import_screens.js'; +import { TOURS, markTourSeen, renderAddMenu, renderBatchHero, renderChooseScreen, seenTours } from './ui_tts_import_screens.js'; /** * The GPT-SoVITS import window, through the Anomalous_TTS node. It first asks what @@ -22,16 +24,19 @@ import { TOURS, markTourSeen, renderBatchHero, renderChooseScreen, seenTours } f * character's "Add files"). A spotlight tour explains each screen the first time. * * Files come from a drop (folders too), the browser's file or folder dialog - * (uploaded in chunks, a few files at a time) or the node's folder picker (sent as - * paths); see ui_tts_import_sources.js. The node copies what is imported; the - * originals are never touched. Clips outside 3–10 s are left out. Each draft is inspected and + * (uploaded in chunks, a few files at a time, once they are in a character) or the + * node's folder picker (sent as paths); see ui_tts_import_sources.js. A large add + * opens a batch of characters at a time; the rest waits for "next batch" (and opens + * by itself once the batch is imported). Files that would clash by name inside a + * character get their folder's name in front. Every clip can be listened to at any + * time (ui_tts_import_player.js). The node copies what is imported; the originals + * are never touched. Clips outside 3–10 s are left out. Each draft is inspected and * imported on its own, so one failure leaves the others alone. Every file is one * row object, owned by exactly one draft's `rows`; its elements are built once * (ui_tts_import_draft.js) and move with it. */ let activeScope = null; -const UPLOADS_AT_ONCE = 3; function el(tag, className, text) { const node = document.createElement(tag); if (className) node.className = className; @@ -86,9 +91,12 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) let nextDraft = 0; let nextRow = 0; let importing = false; - let player = null; - const uploadQueue = []; - let uploading = 0; + const player = createImportPlayer(); + const pending = []; // groups of characters waiting for the next batch (tts_import_groups.js planGroups) + const uploads = createUploadQueue({ + onProgress: updateRow, + onDone: (row) => { updateRow(row); touch(ownerOf(row)); refreshSoon(); }, + }); let refreshTimer = 0; const overlay = el('div', 'anomalous-voice-modal-overlay'); @@ -101,8 +109,9 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) closeSpotlightTour(); for (const draft of drafts) clearTimeout(draft.inspectTimer); clearTimeout(refreshTimer); - uploadQueue.length = 0; - player?.pause(); + uploads.clear(); + pending.length = 0; + player.stop(); const rows = allRows(); for (const row of rows) { row.controller.abort(); @@ -136,7 +145,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) const openDrafts = () => drafts.filter(draft => !draft.done); const solo = () => mode !== 'batch' && drafts.length === 1; - const rowView = row => ({ kind: row.kind, name: row.name, emotion: row.emotion, error: row.error, uploaded: Boolean(row.spec), + const rowView = row => ({ kind: row.kind, name: row.name, emotion: row.emotion, error: row.error, uploaded: Boolean(row.spec), queued: row.queued, progress: row.progress, existing: row.info?.existing ?? null, supported: row.info?.supported, version: row.info?.version, seconds: row.info?.seconds }); /** Keys of the draft's rows the import leaves out (clips outside 3–10 s and their line files). */ @@ -172,6 +181,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) } function forgetRow(row) { + player.forget(row); row.controller.abort(); if (row.uploadId) discardUploads([row.uploadId]); if (row.url) URL.revokeObjectURL(row.url); @@ -187,15 +197,30 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) if (redraw) draw(); } - /** GPT and SoVITS hold one file per draft: a newer one replaces the old. Other kinds skip duplicates. */ + /** What the node gets for a row: its upload or path, and its name when it was renamed. */ + const specOf = row => (row.name !== row.original ? { ...row.spec, name: row.name } : row.spec); + + /** + * GPT and SoVITS hold one file per draft: a newer one replaces the old. The same file + * twice is skipped; another file of the same name is renamed (tts_import_groups.js + * clashFreeNames). In the tray a file keeps its own name and is not uploaded yet. + */ function placeRow(place, row) { if (place !== tray && (row.kind === 'gpt' || row.kind === 'sovits')) { const old = place.rows.find(other => other.kind === row.kind && other !== row); if (old) removeRow(old, { redraw: false }); - } else if (place.rows.some(other => other !== row && other.kind === row.kind && other.name.toLowerCase() === row.name.toLowerCase())) { + } else if (place.rows.some(other => other !== row && other.kind === row.kind && other.dir === row.dir + && other.original.toLowerCase() === row.original.toLowerCase())) { return false; } + row.name = place === tray ? row.original + : clashFreeNames([{ name: row.original, dir: row.dir, kind: row.kind }], place.rows.filter(other => other !== row))[0]; place.rows.push(row); + updateRow(row); + if (place !== tray && row.file && !row.queued) { + row.queued = true; + uploads.add(row); + } return true; } @@ -225,48 +250,35 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) draw(); } - /** - * Add files: `items` are `{ name, dir, file }` or `{ name, dir, path, size }`. Into - * `into` when given (or the only card outside batch mode), else sorted into drafts. - */ - function addItems(items, into = null) { - const known = items.map(item => ({ ...item, kind: importKind(item.name) })); - const bad = known.filter(item => !item.kind).map(item => item.name); - const usable = known.filter(item => item.kind); - const place = into || (solo() ? drafts[0] : null); - let assign; - if (place) { - assign = usable.map(() => place); - } else { - const grouped = groupFiles(usable, openDrafts().map(draft => ({ key: draft.key, target: draft.target }))); - for (const add of grouped.added) newDraft(add); - assign = grouped.assign.map(key => (key === null ? tray : openDrafts().find(draft => draft.key === key))); - } - const notes = []; + /** Put `groups` (planGroups) into their characters: rows, uploads, and what to tell the user. */ + function fill(groups, bad = []) { const keptBest = []; const touched = new Set(); - for (const owner of new Set(assign)) { - const mine = usable.filter((_, i) => assign[i] === owner); - const { skip, kept } = pickWeights(mine); + let renamed = 0; + for (const group of groups) { + const owner = group.owner || (group.tray ? tray + : openDrafts().find(draft => draft.key === group.key) || newDraft({ key: group.key, name: group.name })); + const { skip, kept } = pickWeights(group.items); keptBest.push(...kept); - mine.forEach((item, i) => { + group.items.forEach((item, i) => { if (skip.has(i)) return; const row = { - key: nextRow++, kind: item.kind, name: item.name, dir: item.dir || '', path: item.path || '', file: item.file || null, - size: item.file?.size ?? item.size ?? 0, spec: item.path ? { path: item.path } : null, uploadId: null, progress: 0, - error: '', info: null, emotion: '', text: '', textEdited: false, committed: false, controller: new AbortController(), + key: nextRow++, kind: item.kind, name: item.name, original: item.name, dir: item.dir || '', path: item.path || '', + file: item.file || null, size: item.file?.size ?? item.size ?? 0, spec: item.path ? { path: item.path } : null, + queued: false, uploadId: null, progress: 0, error: '', info: null, emotion: '', text: '', textEdited: false, + committed: false, controller: new AbortController(), }; if (!placeRow(owner, row)) { bad.push(item.name); return; } + if (row.name !== row.original) renamed++; touched.add(owner); - if (row.file) queueUpload(row); }); } - if (keptBest.length > 2) { - notes.push(t('ttsImportKeptBestMany', { count: keptBest.reduce((sum, k) => sum + k.others, 0) })); - } else { - notes.push(...keptBest.map(k => t('ttsImportKeptBest', { kind: t(`ttsKind_${k.kind}`), file: k.name, count: k.others }))); - } + const notes = keptBest.length > 2 + ? [t('ttsImportKeptBestMany', { count: keptBest.reduce((sum, k) => sum + k.others, 0) })] + : keptBest.map(k => t('ttsImportKeptBest', { kind: t(`ttsKind_${k.kind}`), file: k.name, count: k.others })); + if (renamed) notes.push(t('ttsImportRenamed', { count: renamed })); if (bad.length) notes.push(t('ttsImportSkipped', { files: bad.join(', ') })); + if (pending.length) notes.push(t('ttsBatchLater', { count: pending.length })); noteLine.textContent = notes.join(' '); touched.forEach(touch); const first = [...touched].find(owner => owner !== tray); @@ -274,18 +286,27 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) draw(); } - function queueUpload(row) { - uploadQueue.push(row); - pumpUploads(); + /** + * Add files: `items` are `{ name, dir, file }` or `{ name, dir, path, size }`. Into + * `into` when given (or the only card outside batch mode), else sorted into drafts, + * a batch at a time. + */ + function addItems(items, into = null) { + const known = items.map(item => ({ ...item, kind: importKind(item.name) })); + const bad = known.filter(item => !item.kind).map(item => item.name); + const usable = known.filter(item => item.kind); + const place = into || (solo() ? drafts[0] : null); + if (place) { fill([{ owner: place, items: usable }], bad); return; } + const { now, later } = splitBatch(planGroups(usable, openDrafts().map(draft => ({ key: draft.key, target: draft.target })))); + pending.push(...later); + fill(now, bad); } - function pumpUploads() { - while (uploading < UPLOADS_AT_ONCE && uploadQueue.length) { - const row = uploadQueue.shift(); - if (row.controller.signal.aborted) continue; - uploading++; - upload(row).finally(() => { uploading--; pumpUploads(); }); - } + /** Open the next batch of waiting characters. */ + function nextBatch() { + const { now, later } = splitBatch(pending.splice(0)); + pending.push(...later); + fill(now); } /** Many uploads finish close together: repaint once for them. */ @@ -293,24 +314,6 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) if (!refreshTimer) refreshTimer = setTimeout(() => { refreshTimer = 0; if (!scope.signal.aborted) refresh(); }, 100); } - async function upload(row) { - if (row.controller.signal.aborted) return; - try { - await uploadFile(row.file, { - signal: row.controller.signal, - onStart: (id) => { row.uploadId = id; }, - onProgress: (fraction) => { row.progress = fraction; updateRow(row); }, - }); - row.spec = { upload: row.uploadId }; - } catch (e) { - if (row.controller.signal.aborted) return; - row.error = e.message || String(e); - } - updateRow(row); - touch(ownerOf(row)); - refreshSoon(); - } - async function inspect(draft) { if (draft.done || scope.signal.aborted) return; const ready = draft.rows.filter(row => row.spec && !row.error); @@ -322,7 +325,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) return; } try { - const result = await inspectImport(ready.map(row => row.spec), scope.signal, draft.target); + const result = await inspectImport(ready.map(specOf), scope.signal, draft.target); if (mine !== draft.inspectToken || scope.signal.aborted) return; const skippedBefore = skippedKeys(draft.rows).size; ready.forEach((row, i) => { @@ -386,26 +389,16 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) character: (name) => { mode = 'add'; expand(newDraft({ key: groupKey(name), name, target: name })); draw(); }, }); - // "Add more" is one button with a small menu, so the list stays the main thing. - const menu = el('div', 'anomalous-tts-menu'); - const menuItem = (label, hint, onClick) => { - const item = button('anomalous-tts-menu-item', '', () => { menu.classList.remove('is-open'); onClick(); }); - item.append(el('span', 'anomalous-tts-menu-label', label), el('span', 'anomalous-tts-menu-hint', hint)); - return item; - }; - menu.append(menuItem(t('ttsImportPickFiles'), t('ttsMenuUploadHint'), () => chooseFiles()), - menuItem(t('ttsBatchPickFolder'), t('ttsMenuUploadHint'), sources.chooseFolder), - menuItem(t('ttsMenuLocalFiles'), t('ttsMenuLocalHint'), () => chooseLocalFiles()), - menuItem(t('ttsBatchPickLocalFolder'), t('ttsMenuLocalHint'), chooseLocalFolder)); - const addMore = button('anomalous-tts-ghost', t('ttsAddMore'), () => menu.classList.toggle('is-open')); - const menuWrap = el('div', 'anomalous-tts-menu-wrap'); - menuWrap.append(addMore, menu); + const { root: menuWrap, menu } = renderAddMenu({ + files: () => chooseFiles(), folder: sources.chooseFolder, localFiles: () => chooseLocalFiles(), localFolder: chooseLocalFolder, + }); scope.listen(document, 'mousedown', (e) => { if (!menuWrap.contains(e.target)) menu.classList.remove('is-open'); }); const newBtn = button('anomalous-tts-ghost', t('ttsBatchNewDraft'), () => { expand(newDraft({ key: `#new${nextDraft}`, name: '' })); draw(); cardsBox.querySelector('.anomalous-tts-card.is-open .anomalous-tts-card-name-input')?.focus(); }); + const nextBtn = button('anomalous-tts-pick', '', nextBatch); const backBtn = button('anomalous-tts-link', t('ttsChooseAgain'), () => { drafts.length = 0; mode = null; @@ -414,7 +407,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) const toolbar = el('div', 'anomalous-tts-toolbar'); const found = el('span', 'anomalous-tts-toolbar-title'); const dropHint = el('span', 'anomalous-tts-toolbar-hint', t('ttsSoloDropHint')); - toolbar.append(found, dropHint, backBtn, menuWrap, newBtn); + toolbar.append(found, dropHint, backBtn, nextBtn, menuWrap, newBtn); const noteLine = el('div', 'anomalous-tts-note'); const cardsBox = el('div', 'anomalous-tts-cards'); const content = el('div', 'anomalous-tts-content'); @@ -442,12 +435,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) status, signal: scope.signal, places, - play: (row) => { - player?.pause(); - row.url = row.url || URL.createObjectURL(row.file); - player = new Audio(row.url); - player.play().catch(() => {}); - }, + play: row => player.toggle(row), pick: (draft, kinds, local) => (local ? chooseLocalFiles(kinds, draft) : chooseFiles(kinds, draft)), onToggle: (draft) => { if (draft.done) return; @@ -535,6 +523,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) const parts = []; if (waiting && !solo()) parts.push(t('ttsFooterWaiting', { count: waiting })); if (tray.rows.length) parts.push(t('ttsFooterTray', { count: tray.rows.length })); + if (pending.length) parts.push(t('ttsFooterPending', { count: pending.length })); summary.textContent = parts.join(' · '); summary.hidden = !parts.length; saveTo.textContent = t('ttsFooterSaveTo', { path: status.storage }); @@ -568,6 +557,8 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) menuWrap.hidden = !batch; newBtn.hidden = !batch; dropHint.hidden = batch; + nextBtn.hidden = !pending.length; + nextBtn.textContent = t('ttsBatchNext', { count: pending.length }); backBtn.hidden = Boolean(target) || allRows().length > 0; if (content.firstChild !== toolbar) content.replaceChildren(toolbar, noteLine, cardsBox); content.scrollTop = top; @@ -591,7 +582,7 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) const body = buildImportBody({ target: draft.target, library: status.storage, character: draft.name.trim(), language: draft.language, referenceIndex: index >= 0 ? index : null, - rows: ready.map(row => ({ spec: row.spec, kind: row.kind, emotion: row.emotion, text: row.text })), + rows: ready.map(row => ({ spec: specOf(row), kind: row.kind, emotion: row.emotion, text: row.text })), }); try { last = await commitImport(body); @@ -608,6 +599,20 @@ export async function openTtsImport({ files = [], target = null, onDone } = {}) importing = false; if (scope.signal.aborted) return; if (last) onDone?.(last); + if (!openDrafts().length && pending.length) { + // This batch is in: its cards make way and the next batch opens by itself. + const imported = drafts.splice(0); + for (const draft of imported) { + cards.delete(draft); + for (const row of draft.rows) { // committed: the node already moved the uploads + player.forget(row); + if (row.url) URL.revokeObjectURL(row.url); + } + } + nextBatch(); + noteLine.textContent = `${t('ttsBatchImported', { count: imported.length })} ${noteLine.textContent}`; + return; + } if (!openDrafts().length && !tray.rows.length) { scope.dispose(); return; diff --git a/web/modules/ui_tts_import_draft.js b/web/modules/ui_tts_import_draft.js index 7fb01bb..3585fe2 100644 --- a/web/modules/ui_tts_import_draft.js +++ b/web/modules/ui_tts_import_draft.js @@ -51,17 +51,20 @@ function input(className, placeholder) { return node; } -/** State text, size / version / length, and the row's colour. */ +/** Name (a renamed file says what it was), state text, size / version / length, and the row's colour. */ export function updateRow(row) { if (!row.els) return; const info = row.info || {}; + const from = [row.dir, row.original].filter(Boolean).join('/'); + row.els.name.textContent = row.name; + row.els.name.title = row.name !== row.original ? t('ttsImportRenamedFrom', { file: row.path || from }) : row.path || from; const parts = row.size ? [formatSize(row.size)] : []; if (info.version) parts.push(info.supported ? info.version : t('ttsImportUnsupported', { version: info.version })); if (info.seconds !== undefined) parts.push(t('ttsImportSeconds', { seconds: info.seconds })); row.els.meta.textContent = parts.join(' · '); const outOfRange = row.kind === 'audio' && info.seconds !== undefined && !usableAsReference(info.seconds); row.els.root.classList.toggle('is-out-of-range', outOfRange); - const view = rowState({ error: row.error, uploaded: Boolean(row.spec), progress: row.progress, + const view = rowState({ error: row.error, uploaded: Boolean(row.spec), queued: row.queued, progress: row.progress, existing: info.existing ?? null, unsupported: info.supported === false }); row.els.root.dataset.tone = view.tone; const leftOut = outOfRange && view.tone === 'ready'; @@ -122,8 +125,7 @@ function rowElements(row, ctx) { if (row.els) return row.els; const root = el('div', `anomalous-tts-file is-${row.kind}`); const name = el('span', 'anomalous-tts-file-name', row.name); - name.title = row.path || row.name; - const els = { root, meta: el('span', 'anomalous-tts-file-meta'), state: el('span', 'anomalous-tts-file-state'), + const els = { root, name, meta: el('span', 'anomalous-tts-file-meta'), state: el('span', 'anomalous-tts-file-state'), move: el('select', 'anomalous-tts-file-move') }; row.els = els; els.move.title = t('ttsBatchMoveTo'); @@ -137,8 +139,7 @@ function rowElements(row, ctx) { const label = el('span', 'anomalous-tts-file-label'); label.append(name, els.meta); if (row.kind === 'audio') { - const play = button('anomalous-tts-file-play', '▶', () => ctx.play(row), t('audioPlay')); - play.hidden = !row.file; // local paths cannot be played from the browser + els.play = button('anomalous-tts-file-play', '▶', () => ctx.play(row), t('audioPlay')); els.main = button('anomalous-tts-file-main', t('ttsImportMain'), () => ctx.onReference(row)); els.emotion = input('anomalous-tts-file-emotion', t('ttsImportEmotionPlaceholder')); els.emotion.value = row.emotion; @@ -148,7 +149,7 @@ function rowElements(row, ctx) { els.text.oninput = () => { row.text = els.text.value.trim(); row.textEdited = true; showSource(row, null); }; if (row.info && !row.textEdited) showSource(row, row.info.text_source || 'none'); const top = el('div', 'anomalous-tts-file-top'); - top.append(play, label, els.state, els.main, els.move, remove); + top.append(els.play, label, els.state, els.main, els.move, remove); const bottom = el('div', 'anomalous-tts-file-bottom'); bottom.append(els.emotion, els.text); root.append(top, bottom); diff --git a/web/modules/ui_tts_import_player.js b/web/modules/ui_tts_import_player.js new file mode 100644 index 0000000..5b4380d --- /dev/null +++ b/web/modules/ui_tts_import_player.js @@ -0,0 +1,59 @@ +import { t } from './interface_settings.js'; +import { previewUrl } from './tts_setup_api.js'; + +/** + * Listening to clips in the import window, while they upload or wait unassigned: + * one clip at a time, its row marked and its button turned into "stop"; clicking + * another clip switches to it, so clips can be compared one after another. Browser + * files play from memory (`row.url`, an object URL the window revokes with the row); + * local paths play through the node (`/anomalous_tts/import/preview`). + */ +export function createImportPlayer() { + let audio = null; + let current = null; + + const mark = (row, playing, error = '') => { + const button = row?.els?.play; + if (!button) return; + button.textContent = playing ? '■' : '▶'; + button.title = error ? t('ttsPreviewFailed', { error }) : t(playing ? 'ttsPreviewStop' : 'audioPlay'); + button.setAttribute('aria-label', button.title); + button.classList.toggle('is-failed', Boolean(error)); + row.els.root.classList.toggle('is-playing', playing); + }; + + function stop() { + audio?.pause(); + audio = null; + mark(current, false); + current = null; + } + + /** Play `row`, or stop it when it is the one playing. */ + function toggle(row) { + if (current === row) { stop(); return; } + stop(); + if (!row.file && !row.path) return; + if (row.file) row.url = row.url || URL.createObjectURL(row.file); + const mine = new Audio(row.file ? row.url : previewUrl(row.path)); + audio = mine; + current = row; + mark(row, true); + const fail = (error) => { + if (audio !== mine) return; + audio = null; + current = null; + mark(row, false, error); + }; + mine.onended = () => { if (audio === mine) stop(); }; + mine.onerror = () => fail(mine.error?.message || t('ttsPreviewUnreadable')); + mine.play().catch(e => fail(e.message || String(e))); + } + + return { + toggle, + stop, + /** A row leaves the window: stop it if it is playing. */ + forget: (row) => { if (current === row) stop(); }, + }; +} diff --git a/web/modules/ui_tts_import_screens.js b/web/modules/ui_tts_import_screens.js index 01e339b..d366997 100644 --- a/web/modules/ui_tts_import_screens.js +++ b/web/modules/ui_tts_import_screens.js @@ -2,8 +2,9 @@ import { t } from './interface_settings.js'; /** * The fixed screens of the GPT-SoVITS import window (ui_tts_import.js): the first - * question (what do you have?), the drop area of batch mode, and the spotlight tour - * steps that explain them. Only markup and callbacks; the window owns all state. + * question (what do you have?), the drop area of batch mode, the "add more" menu, + * and the spotlight tour steps that explain them. Only markup and callbacks; the + * window owns all state. */ const UPLOAD_SVG = ''; @@ -108,3 +109,24 @@ export function renderBatchHero(on) { el('div', 'anomalous-tts-hero-desc', t('ttsHeroDesc')), buttons, local, button('anomalous-tts-link', t('ttsChooseAgain'), on.back)); return hero; } + +/** + * "Add more": one button with a small menu, so the list stays the main thing. + * `on`: `{ files, folder, localFiles, localFolder }`. Returns `{ root, menu }`; the + * window closes the menu on an outside click or Escape (`menu` loses `is-open`). + */ +export function renderAddMenu(on) { + const menu = el('div', 'anomalous-tts-menu'); + const item = (label, hint, onClick) => { + const node = button('anomalous-tts-menu-item', '', () => { menu.classList.remove('is-open'); onClick(); }); + node.append(el('span', 'anomalous-tts-menu-label', label), el('span', 'anomalous-tts-menu-hint', hint)); + return node; + }; + menu.append(item(t('ttsImportPickFiles'), t('ttsMenuUploadHint'), on.files), + item(t('ttsBatchPickFolder'), t('ttsMenuUploadHint'), on.folder), + item(t('ttsMenuLocalFiles'), t('ttsMenuLocalHint'), on.localFiles), + item(t('ttsBatchPickLocalFolder'), t('ttsMenuLocalHint'), on.localFolder)); + const root = el('div', 'anomalous-tts-menu-wrap'); + root.append(button('anomalous-tts-ghost', t('ttsAddMore'), () => menu.classList.toggle('is-open')), menu); + return { root, menu }; +} diff --git a/web/modules/ui_tts_import_sources.js b/web/modules/ui_tts_import_sources.js index 3caea79..58f4f2e 100644 --- a/web/modules/ui_tts_import_sources.js +++ b/web/modules/ui_tts_import_sources.js @@ -1,15 +1,18 @@ import { t } from './interface_settings.js'; import { anomalousAlert, anomalousConfirm } from './ui_dialog.js'; import { importKind, scanFolder } from './tts_setup_api.js'; -import { isSkippedFolder } from './tts_import_groups.js'; +import { DROP_FILE_LIMIT, sortOutFolders } from './tts_import_groups.js'; import { formatSize, pickServerPath } from './ui_tts_path_picker.js'; /** * Where the import window's files come from: the browser's file and folder dialogs * and drops (uploaded to the node), or the node's own picker (local paths, copied - * without uploading). Files inside GPT-SoVITS program or training folders are left - * out, and a very large upload first offers the node's picker instead: ComfyUI runs - * on this computer, so copying from the path skips sending every byte twice. + * without uploading). Folders of Python environments, base models and a GPT-SoVITS + * package's program and training runs are left out (tts_import_groups.js skipFolder), + * and the user is told which; a very large upload first offers the node's picker + * instead: ComfyUI runs on this computer, so copying from the path skips sending + * every byte twice. Every source gives each file's folder starting with the folder + * that was chosen, so files are sorted the same whichever way they came in. */ const ACCEPT = { gpt: '.ckpt', sovits: '.pth', audio: '.wav,.flac,.ogg,.mp3', text: '.txt,.lab,.list' }; @@ -17,11 +20,10 @@ const ACCEPT_ALL = Object.values(ACCEPT).join(','); export const UPLOAD_WARN_BYTES = 2 * 1024 ** 3; export const UPLOAD_WARN_FILES = 1500; -/** The first skipped folder on `dir` (`a/logs/b` → `a/logs`), or null. */ -function skippedPart(dir) { - const parts = String(dir || '').split('/'); - const at = parts.findIndex(isSkippedFolder); - return at < 0 ? null : parts.slice(0, at + 1).join('/'); +/** `a、b、c…`: the first few names of a list, in name order. */ +function few(names) { + const sorted = [...names].sort((a, b) => a.localeCompare(b)); + return sorted.slice(0, 3).join('、') + (sorted.length > 3 ? '…' : ''); } /** @@ -41,7 +43,11 @@ export function createFileSources({ signal, add, note }) { if (!path || signal.aborted) return; try { const result = await scanFolder(path, signal); - add(result.files.map(file => ({ name: file.name, dir: file.dir, path: file.path, size: file.size }))); + const top = result.path.split('/').filter(Boolean).pop() || ''; + const under = dir => [top, dir].filter(Boolean).join('/'); + add(result.files.map(file => ({ name: file.name, dir: under(file.dir), path: file.path, size: file.size }))); + if (result.skipped?.length) note(t('ttsSkippedFolders', { folders: few(result.skipped.map(under)) })); + if (result.too_deep?.length) note(t('ttsScanTooDeep', { folders: few(result.too_deep.map(under)) })); if (result.truncated) note(t('ttsBatchScanTruncated', { count: result.files.length })); } catch (e) { if (!signal.aborted) await anomalousAlert(t('ttsSetupFailed', { error: e.message || String(e) })); @@ -49,18 +55,14 @@ export function createFileSources({ signal, add, note }) { }; /** - * Browser files, `{ file, dir }` (or `{ skipped: folder }` for a folder a drop did - * not go into). Leaves out program and training folders, and asks before a very - * large upload. + * Browser files, `{ file, dir }`, with `{ skipped: folder }` and `{ truncated: true }` + * markers from a drop (ui_tts_file_drop.js). Says what was left out, and asks before + * a very large upload. */ async function takeFiles(list, into = null) { - const skipped = new Set(); - const items = []; - for (const item of list) { - const part = item.skipped || skippedPart(item.dir); - if (part) skipped.add(part); - else items.push({ name: item.file.name, dir: item.dir || '', file: item.file }); - } + const skipped = list.filter(item => item.skipped).map(item => item.skipped); + const truncated = list.some(item => item.truncated); + const items = list.filter(item => item.file).map(item => ({ name: item.file.name, dir: item.dir || '', file: item.file })); const usable = items.filter(item => importKind(item.name)); const bytes = usable.reduce((sum, item) => sum + item.file.size, 0); if (bytes > UPLOAD_WARN_BYTES || usable.length > UPLOAD_WARN_FILES) { @@ -70,10 +72,17 @@ export function createFileSources({ signal, add, note }) { if (choice) { await chooseLocalFolder(); return; } } if (items.length) add(items, into); - if (skipped.size) { - const names = [...skipped]; - note(t('ttsSkippedFolders', { folders: names.slice(0, 3).join('、') + (names.length > 3 ? '…' : '') })); - } + if (skipped.length) note(t('ttsSkippedFolders', { folders: few(skipped) })); + if (truncated) note(t('ttsBatchScanTruncated', { count: items.length })); + } + + /** The browser's folder dialog lists every file: leave out the same folders a drop does. */ + function takeFolder(files) { + const all = files.map(file => ({ file, name: file.name, dir: file.webkitRelativePath.split('/').slice(0, -1).join('/') })); + const { kept, skipped } = sortOutFolders(all); + const usable = kept.map(i => all[i]).filter(item => importKind(item.name)); + const list = usable.slice(0, DROP_FILE_LIMIT).map(item => ({ file: item.file, dir: item.dir })); + takeFiles([...list, ...skipped.map(folder => ({ skipped: folder })), ...(usable.length > DROP_FILE_LIMIT ? [{ truncated: true }] : [])]); } const fileInput = document.createElement('input'); @@ -89,7 +98,7 @@ export function createFileSources({ signal, add, note }) { folderInput.webkitdirectory = true; folderInput.hidden = true; folderInput.onchange = () => { - takeFiles([...folderInput.files].map(file => ({ file, dir: file.webkitRelativePath.split('/').slice(0, -1).join('/') }))); + takeFolder([...folderInput.files]); folderInput.value = ''; }; const chooseFiles = (kinds = null, into = null) => { diff --git a/web/modules/ui_tts_import_uploads.js b/web/modules/ui_tts_import_uploads.js new file mode 100644 index 0000000..7ea837f --- /dev/null +++ b/web/modules/ui_tts_import_uploads.js @@ -0,0 +1,45 @@ +import { uploadFile } from './tts_setup_api.js'; + +/** + * The import window's uploads: browser files go to the node in chunks, a few files at + * a time, in the order they were queued. A row is `{ file, controller, uploadId, + * progress, spec, error }` (ui_tts_import.js); an aborted row is dropped. `onProgress(row)` + * repaints the row; `onDone(row)` runs once it is uploaded (`spec` set) or failed (`error`). + */ + +const UPLOADS_AT_ONCE = 3; + +export function createUploadQueue({ onProgress, onDone }) { + const queue = []; + let running = 0; + + async function upload(row) { + try { + await uploadFile(row.file, { + signal: row.controller.signal, + onStart: (id) => { row.uploadId = id; }, + onProgress: (fraction) => { row.progress = fraction; onProgress(row); }, + }); + row.spec = { upload: row.uploadId }; + } catch (e) { + if (row.controller.signal.aborted) return; + row.error = e.message || String(e); + } + onDone(row); + } + + function pump() { + while (running < UPLOADS_AT_ONCE && queue.length) { + const row = queue.shift(); + if (row.controller.signal.aborted) continue; + running++; + upload(row).finally(() => { running--; pump(); }); + } + } + + return { + add: (row) => { queue.push(row); pump(); }, + /** The window closes: nothing more starts (running uploads stop through their rows' controllers). */ + clear: () => { queue.length = 0; }, + }; +} diff --git a/web/styles/12-tts-setup.css b/web/styles/12-tts-setup.css index 081b399..4e87e74 100644 --- a/web/styles/12-tts-setup.css +++ b/web/styles/12-tts-setup.css @@ -988,6 +988,7 @@ .anomalous-tts-file[data-tone="busy"] { border-left-color: #38bdf8; } .anomalous-tts-file[data-tone="skip"] { border-left-color: #64748b; opacity: 0.7; } +.anomalous-tts-file[data-tone="idle"] { border-left-color: #475569; } .anomalous-tts-file[data-tone="error"] { border-color: rgba(248, 113, 113, 0.4); @@ -1032,6 +1033,24 @@ color: #fff; } +/* The clip being listened to: its button is "stop". */ +.anomalous-tts-file.is-playing { + border-color: rgba(129, 140, 248, 0.45); + background: rgba(129, 140, 248, 0.08); +} + +.anomalous-tts-file.is-playing .anomalous-tts-file-play { + padding: 0; + border-color: #818cf8; + background: #818cf8; + color: #fff; +} + +.anomalous-tts-file-play.is-failed { + border-color: rgba(248, 113, 113, 0.6); + color: #f87171; +} + .anomalous-tts-file-label { display: flex; flex: 1 1 auto;