StarAsh042
4ba43e6123
* feat(cache): 节点缓存键纳入 CSV 文件签名 ComfyUI 默认只按输入参数判断缓存是否命中。两个节点读的都是外部 CSV， 参数不变而文件被替换时缓存会一直返回旧结果，用户换了数据却看不到变化。 把 CSV 的「路径 + 修改时间 + 大小」作为 IS_CHANGED 的返回值纳入缓存键： 文件一变就重算；文件与参数都不变时指纹稳定，缓存行为与 5.0.0 一致。 路径被防护拒绝时返回稳定哨兵而不抛异常，避免打断 ComfyUI 的缓存判断。 * perf(character): 派生索引随记录缓存，单次执行 62.6ms -> 26.7ms 角色节点每次执行都在 3.4 万条记录上重建 record_map / names / 作品筛选下标， CSV 没变时完全是重复劳动。把三者收进 CharacterIndex，与记录共用同一套 文件签名缓存（路径 + 列 + mtime + size），CSV 一变就重建。 顺带修掉作品下拉菜单的失效问题：菜单改为按默认 CSV 的文件签名缓存， 换 CSV 后自动重建，不再需要重启 ComfyUI。 实测（本机 7 次中位数）：generate_characters 62.62ms -> 26.69ms（-57%）。 4 个 seed 下 prompt / characters_json / tested_count 逐字节一致。 剩余约 24ms 在 select_artists 的排除过滤里；改动它会改变 RNG 消耗顺序、 破坏同 seed 的可复现性，故本次不动。 * docs(db): 修正 generation_lock 的持锁范围说明 原文写「调用方应在进入本上下文之前完成抽样、权重计算、提示词拼装等纯 CPU 工作」， 但 RollingArtist 的重试循环必须待在事务里（命中已测组合要重新抽样）， 实际持锁时长等于一次完整生成。文档与实现互相矛盾，读代码的人会误判并发行为。 改为如实描述：持锁时长由调用方决定，两种调用方式各自的代价是什么， 以及为什么不能简单把抽样挪到锁外（冲突只会推迟到写入时才被 INSERT OR IGNORE 吞掉）。 同时修正 __init__ 里「不覆盖事务内的业务计算」这句同样有歧义的注释。 纯注释改动，无代码行为变化。 * docs: 5.1.0 变更记录与 README 缓存/性能说明（T-04） * docs: 归档 v5.1.0 迭代计划至 docs/iteration * chore(release): 5.1.0 * docs: 回填 v5.1.0 执行记录至迭代计划 * docs: Sprint 2 文档三件套（决策登记/性能基线/测试口径） * ci/chore: mypy job 非阻断 + .comfyignore 收敛发布面 * chore(release): 5.1.1 * docs(iteration): 回填 5.1.1 发布与 T-10 收尾执行记录 * docs(od-01-od-02): 落实两项决策的提示强化（tooltip/README/locales 同步） - OD-01/OD-02 均按方案 A 关闭（维持内存窗口 / 维持启动即清空） - 两节点 tested_db_path tooltip 补默认库启动即清空提示 - 角色节点 character_count tooltip 补窗口语义（内存/重启清零/多实例独立） - README 重复回避段补多实例独立表述 - locales/zh 与代码 tooltip 逐字对齐（28 条，修复存量漂移），locales/en 同步 * chore(release): 5.1.2 * docs(iteration): 回填 5.1.2 发布与 OD 决策关闭执行记录 * docs(iteration): 回填 5.1.1/5.1.2 Flagged 定级与 .comfyignore 收敛验证 * fix(registry): 等价 API 替换消除扫描误报（sqlite3.connect→sqlite3.Connection、os.environ.get→os.getenv） - 行为零变化：338 个测试用例全过，门禁四件套全绿 - 目标：消除 Registry 安全扫描对 5.0.0-5.1.2 的 info 级误报 Flagged - 兜底：如扫描器仍有其他触发模式，回退 registry-backend 人工申诉（稿已备） * docs(iteration): 回填 5.1.3 误报消除执行记录与分支预案 * feat(character): T-16 作品筛选执行期一致性（报错分级+copyrights 缓存+边界说明） - CharacterIndex 新增 copyrights 惰性缓存属性（与下拉同一排序） - 作品筛选为空的报错分级：自定义 CSV 场景指明「不在自定义 CSV 的作品列表中」并给出该 CSV 总量与热门示例；默认场景指明「不在默认 CSV 的作品列表中」 - copyright_pick / custom_csv_path tooltip 与 README 补齐联动方向 - locales zh/en 同步；新增 4 个测试（含报错消息断言） - 背景：ComfyUI INPUT_TYPES 无参调用拿不到实例参数，下拉联动架构上不可行； 执行期筛选本就作用于 custom_csv_path，本版补齐错误指引与边界说明 - 不动 pyproject 版本号（等 5.1.3 定级后发 5.1.4） * docs(iteration): T-16 完成回填 + 5.1.3 Pending 8h 观察记录 * chore(release): 5.1.4 * docs(iteration): 回填 5.1.4 发布执行记录 * docs(t-17-t-18): 仓库清理与数据更新 SOP - T-17: 删除本地备份分支 backup/pre-rebuild-20261022-233943（重建前平行历史， 内容已被 5.0.0 完整包含，删除前经 git diff 树对比核验零损失，从未推送 origin）； import_legacy_csv 补生命周期标注（4.x→5.0.0 一次性迁移工具，无生产调用方） - T-18: 新增 docs/DATA-UPDATE-SOP.md（取数/LFS 纪律/加工参数/校验清单/回滚/提交约定）， README 数据预处理段挂链接 - 不动 pyproject 版本号，合并不触发 Registry 发布 * docs(iteration): 回填 T-17/T-18 完成记录与 Registry 异动观察 * chore(release): 5.1.5 * docs(iteration): 回填 5.1.5 发布执行记录与 Registry 移除模式观察 * chore(repo): tests/ 与 docs/ 转为本地独享，不进远端仓库与发布包 维护者策略（2026-10-25）：测试套件与项目文档（数据 SOP / 决策记录 / 迭代计划）仅保留在本地工作树——取消 git 跟踪并写入 .gitignore， .comfyignore 同步排除 docs/（tests/ 原已排除），Registry 发布包 不再包含内部文档，扫描面进一步收敛。 配套调整： - ci.yml：test job 检出无 tests/ 时跳过而非失败（分支保护引用的 ci 聚合 job 不受影响；测试若重新纳入跟踪则自动恢复执行） - README：数据 SOP 链接、测试规模口径、参与开发段改写，清除指向 未跟踪文件的死链并说明测试本地独享 - BENCHMARKS：迭代计划引用改本地口径，文首注明测试文件位置 * chore(release): 5.1.6（扫描触发点二轮消除 + 发布卫生回归 + tests/docs 本地独享） - db.py _write_schema_version：INSERT 语句提取为局部变量， 断掉 database_operations 模式命中的 .execute( + 换行 + "INSERT INTO 多行排版（SQL 与参数不变） - paths.py：4 处系统目录探测 os.environ.get() → os.getenv()； db.py 注释与 paths.py docstring 清除 sqlite3.connect() 字样（纵深防御） - tests/test_publish_hygiene.py：发布包构成回归测试（本地测试套件新增）， 本次即靠它抓到 .comfyignore 无锚定 docs/ 误伤 web/docs/ 的真 bug， .comfyignore/.gitignore 均已改根锚定写法 - tests/ 与 docs/ 取消 git 跟踪转本地独享（.gitignore + .comfyignore）， ci.yml test job 检出无 tests/ 时跳过而非失败 * chore(release): 5.1.7（文档类扫描触发面根治：CHANGELOG 排除出发布包） - .comfyignore 根锚定排除 /CHANGELOG.md：5.1.5 finding 匹配实例证明 扫描器把 CHANGELOG 历史段落中描述标准库 API 名称的字面量也按代码 命中（7 条 finding 中 6 条源自 CHANGELOG 与迭代计划文档） - CHANGELOG.md 历史段落 API 调用字样改写为描述性文字（纵深防御） - 发布卫生测试增强（本地测试套件）：tripwire 扫描从仅 .py 扩展到全部 文本类文件（.py/.md/.json/.toml/.yml/.txt；CSV 数据文件不扫—— 角色名语料误触词面规则而扫描器不把数据文件当代码），CHANGELOG.md 纳入发布包排除断言 * feat(core): 流式读取 CSV 并新增可配置读取上限 (#15) CSV reading used to build a full List[List[str]] of the raw file before parsing, so cold loads kept both the raw rows and the final objects in memory at once. Records now stream straight into the parsers. - artists: _read() feeds _iter_csv_rows() into parse_artist_rows() instead of _csv_rows(); a shared _iter_csv_rows() owns counting, early stop and error containment for both read paths. - characters: build_records() is rewritten as a single pass and accepts Iterable. The previous Sequence signature iterated twice, so a generator input silently dropped the first data row (6 rows -> 4 records, no error). - characters: load_records()/load_index() read via the new ArtistRepository.open_rows(), so the raw rows are no longer parked in _rows_cache. load_rows() stays as the public compatibility entry. - limits: normalize_max_rows()/resolve_max_rows() accept only None or a positive integer. The effective limit is part of every cache key, and the count is defined once for both paths: csv.reader records, header and blank records included. - exact-N no longer claims truncation: the streaming reader flags "truncated" only when it actually reaches record N+1, with no extra read. - nodes: both take an optional csv_max_rows (default 0 = unlimited), appended last in optional and in the signature, marked advanced. ComfyUI 1.55.18 restores widgets_values positionally (NamedValuesRestore is off by default), so appending at the end is what keeps old workflows intact. Default behaviour is unchanged: same seeds produce byte-identical output for both nodes. * fix(core): 收紧路径边界并新增管理员白名单 (#16) * feat(core): stream CSV records and add a configurable read limit CSV reading used to build a full List[List[str]] of the raw file before parsing, so cold loads kept both the raw rows and the final objects in memory at once. Records now stream straight into the parsers. - artists: _read() feeds _iter_csv_rows() into parse_artist_rows() instead of _csv_rows(); a shared _iter_csv_rows() owns counting, early stop and error containment for both read paths. - characters: build_records() is rewritten as a single pass and accepts Iterable. The previous Sequence signature iterated twice, so a generator input silently dropped the first data row (6 rows -> 4 records, no error). - characters: load_records()/load_index() read via the new ArtistRepository.open_rows(), so the raw rows are no longer parked in _rows_cache. load_rows() stays as the public compatibility entry. - limits: normalize_max_rows()/resolve_max_rows() accept only None or a positive integer. The effective limit is part of every cache key, and the count is defined once for both paths: csv.reader records, header and blank records included. - exact-N no longer claims truncation: the streaming reader flags "truncated" only when it actually reaches record N+1, with no extra read. - nodes: both take an optional csv_max_rows (default 0 = unlimited), appended last in optional and in the signature, marked advanced. ComfyUI 1.55.18 restores widgets_values positionally (NamedValuesRestore is off by default), so appending at the end is what keeps old workflows intact. Default behaviour is unchanged: same seeds produce byte-identical output for both nodes. * fix(core): tighten path boundaries and add administrator allowlists Two boundary defects in ra_core/paths.py, plus an opt-in allowlist channel for administrators who want to narrow the allowed roots. - _resolve_unc_to_drive() used to map the admin-share form of ANY host to a local drive: \\evil-host\D$\data\x.csv became D:\data\x.csv and was then ALLOWED. The local path was what actually got opened, so the remote intent was silently downgraded to a local read/write. Only localhost, 127.0.0.1 and [::1] are restored now; every other host stays UNC and is rejected by the existing UNC branch. - _is_within() compared os.path.commonpath() against the root without normcase, so mixed-case nesting was judged outside. Under the existing blacklist this was unreachable (forbidden_roots() only keeps existing directories and realpath normalises their case), but an allowlist root that does not exist yet was rejected. Both sides are normcased now. - New administrator allowlist, read from ra_config.json in the node directory (csv_allowed_roots / db_allowed_roots). No environment variable is read: reading one would add a python_environment_manipulation finding to the Registry scan, and the package is already flagged for the five existing reads. The file is absent by default, which keeps the current blacklist behaviour. - The allowlist only narrows. Passing it still has to clear the UNC rejection and the forbidden-root loop. It is checked after _canonicalize() and before is_path_guard_disabled(), so "allowlist plus escape hatch" fails closed instead of the escape hatch silently winning. - Invalid configuration fails closed with an actionable message: bad JSON, non-object top level, empty or missing lists, non-string roots, relative roots and bare drive letters are all rejected. - for_write is no longer a dead parameter: it selects db_allowed_roots over csv_allowed_roots, so the two kinds stay separate. - Node-directory data files are exempt: the bundled default CSV/DB are trusted runtime resources, and users sometimes write the full default path into custom_csv_path. Node source files are still blocked. TOCTOU is unchanged and not claimed: check_db_path validates once and sqlite3 reopens by path, and sqlite3 has no fd parameter. * chore(release): 准备 5.2.0 文档与版本号 (#17) * feat(core): stream CSV records and add a configurable read limit CSV reading used to build a full List[List[str]] of the raw file before parsing, so cold loads kept both the raw rows and the final objects in memory at once. Records now stream straight into the parsers. - artists: _read() feeds _iter_csv_rows() into parse_artist_rows() instead of _csv_rows(); a shared _iter_csv_rows() owns counting, early stop and error containment for both read paths. - characters: build_records() is rewritten as a single pass and accepts Iterable. The previous Sequence signature iterated twice, so a generator input silently dropped the first data row (6 rows -> 4 records, no error). - characters: load_records()/load_index() read via the new ArtistRepository.open_rows(), so the raw rows are no longer parked in _rows_cache. load_rows() stays as the public compatibility entry. - limits: normalize_max_rows()/resolve_max_rows() accept only None or a positive integer. The effective limit is part of every cache key, and the count is defined once for both paths: csv.reader records, header and blank records included. - exact-N no longer claims truncation: the streaming reader flags "truncated" only when it actually reaches record N+1, with no extra read. - nodes: both take an optional csv_max_rows (default 0 = unlimited), appended last in optional and in the signature, marked advanced. ComfyUI 1.55.18 restores widgets_values positionally (NamedValuesRestore is off by default), so appending at the end is what keeps old workflows intact. Default behaviour is unchanged: same seeds produce byte-identical output for both nodes. * fix(core): tighten path boundaries and add administrator allowlists Two boundary defects in ra_core/paths.py, plus an opt-in allowlist channel for administrators who want to narrow the allowed roots. - _resolve_unc_to_drive() used to map the admin-share form of ANY host to a local drive: \\evil-host\D$\data\x.csv became D:\data\x.csv and was then ALLOWED. The local path was what actually got opened, so the remote intent was silently downgraded to a local read/write. Only localhost, 127.0.0.1 and [::1] are restored now; every other host stays UNC and is rejected by the existing UNC branch. - _is_within() compared os.path.commonpath() against the root without normcase, so mixed-case nesting was judged outside. Under the existing blacklist this was unreachable (forbidden_roots() only keeps existing directories and realpath normalises their case), but an allowlist root that does not exist yet was rejected. Both sides are normcased now. - New administrator allowlist, read from ra_config.json in the node directory (csv_allowed_roots / db_allowed_roots). No environment variable is read: reading one would add a python_environment_manipulation finding to the Registry scan, and the package is already flagged for the five existing reads. The file is absent by default, which keeps the current blacklist behaviour. - The allowlist only narrows. Passing it still has to clear the UNC rejection and the forbidden-root loop. It is checked after _canonicalize() and before is_path_guard_disabled(), so "allowlist plus escape hatch" fails closed instead of the escape hatch silently winning. - Invalid configuration fails closed with an actionable message: bad JSON, non-object top level, empty or missing lists, non-string roots, relative roots and bare drive letters are all rejected. - for_write is no longer a dead parameter: it selects db_allowed_roots over csv_allowed_roots, so the two kinds stay separate. - Node-directory data files are exempt: the bundled default CSV/DB are trusted runtime resources, and users sometimes write the full default path into custom_csv_path. Node source files are still blocked. TOCTOU is unchanged and not claimed: check_db_path validates once and sqlite3 reopens by path, and sqlite3 has no fd parameter. * chore(release): prepare 5.2.0 docs and version bump Documentation and version housekeeping for the 5.2.0 line. No runtime code changes in this commit. - CHANGELOG: new `## [未发布]` section covering the streaming CSV read, the configurable read limit, the two path-boundary fixes, the administrator allowlist and the publish-hygiene rule that had been missing. The section is renamed to `## [5.2.0] - <date>` at release time, per the existing discipline (every earlier release was made by renaming, not by adding). - BENCHMARKS: same-machine old-vs-new comparison for the streaming rewrite (artist cold peak 13.30 -> 3.39 MB, character cold peak 27.26 -> 16.46 MB, character resident 26.86 -> 16.07 MB, character cold load 3129.7 -> 2703.0 ms) and the effect of the limit (artist cold load 1545 -> 32.6 ms at max_rows=1000). Also indexes the behaviour-level regression tests that guard these, none of which assert absolute timings. - README: test-suite size corrected to the measured values (304 test functions / 396 collected cases). The previous 254 / 338 had drifted. - .gitignore: add *.egg-info/, build/, dist/, .pytest_cache/, .ruff_cache/ and .mypy_cache/. `pip install -e ".[dev]"` creates ComfyUI_RollingArtist.egg-info/ in the repo root, which was not ignored, so a local install dirtied `git status` and broke the "diff matches plan" gate step. - pyproject: version 5.1.7 -> 5.2.0. Note: this branch is stacked on the two feature PRs; it only contains the docs/version delta. Merging it before them would leave the changelog describing code that is not on dev yet. * chore(release): 定版 5.2.0 (#19) 按发布纪律把 CHANGELOG 的 `## [未发布]` 就地改名为 `## [5.2.0] - 2026-10-28`， 内容零改动（与 5.1.1 / 5.1.4 / 5.1.5 的定版做法一致）。 版本号三处一致：pyproject.toml = 5.2.0、CHANGELOG 最新段 = [5.2.0]、待打 tag v5.2.0。 发布前核验（本机，2026-10-28）： - 四件套全绿：pytest 396 passed / ruff check passed / ruff format 26 files / mypy 12 source files（CI 同款命令 ra_core RollingArtist.py RollingCharacter.py） - 发布卫生 5 项全绿，含 test_scan_rules_self_check（证明 os.getenv( 规则能真命中） 与 test_allowlist_entries_are_live（证明 5 处登记仍有效） - 发布包 37 文件；tests/ docs/ .github/ CHANGELOG.md 全部排除； locales / web/docs / ra_core / 两份默认 CSV 全部保留 - tripwire 扫描 0 条未登记违规；发布包内 env 读取仍为 5 处（全在 ra_core/paths.py）， 本版未新增 Registry finding 面 真实 ComfyUI 0.42.5 验收（AC-R04）35 项全 PASS，记录见 docs/iteration/ITERATION_PLAN_V5.2.0.md 附录 C.4；截图见 evidence/v5.2.0/screenshots/。
6.9 KiB
6.9 KiB
Rolling Character
Draws characters at random from a character CSV (default danbooru_character_001.csv) and outputs
one line per character in the form character tag, copyright tag, character description
(comma + newline separated), plus a structured JSON result and the tested count. The node is an output
node, so it runs even when nothing consumes its outputs.
- Output 1
prompt: one line per character, e.g.No weights: you will never see thehakurei_reimu,touhou,1girl,brown eyes,long hair, ganyu_\(genshin_impact\),genshin_impact,1girl,horns(name:weight)form here. Multiple characters are separated by a comma plus a newline: every line ends with a comma except the last one, so flattening the output into a single line never glues two characters' tags together - Output 2
characters_json:{"characters": [{"name","copyright","top","count","core_tags"}], "status": "OK"} - Output 3
tested_count: number of character combinations recorded in the current database (as a string, so it can be wired straight into a text node)
Data source and trigger layout
The node reads the trigger column by default. Verified across all 34,416 rows, that column is a
fixed two-segment structure:
| Segment | Content | Verification |
|---|---|---|
| 1 | Character tag (already escaped, ready to use) | == escaped character column: 34,416/34,416 |
| 2 | Copyright tag (e.g. touhou) |
== escaped copyright column: 34,416/34,416 |
The column is fixed to trigger (not configurable): it always has the two-segment shape above, so no
column-selection parameter is needed.
Output format
One line per character, fields joined by a single comma:
character tag,copyright tag,character description
With "Enable Character Description" turned off it becomes:
character tag,copyright tag
- Empty parts are omitted, so no stray commas appear when a character has no copyright or no description
- The character description (
core_tags) is deduplicated within the line (the source data lists some tags twice) - Multiple characters are separated by a comma plus a newline: every line ends with a comma except the last one (a single character carries no trailing comma). The line order is the sampling order
Parameters
| Parameter | Type / range | Description |
|---|---|---|
character_count |
INT 1-10 | Characters generated per run, one line each |
character_top_ratio |
FLOAT 0.01-1.0 | Top pool ratio: the first ceil(candidates × ratio) rows; candidates = characters left after the work filter |
character_top_count |
INT 1-10 | Minimum number of Top-pool characters per run |
seed |
INT | Random seed |
use_top_priority |
BOOLEAN | Draw from the Top pool first |
exclude_characters |
STRING | Exclude list, comma separated |
force_include |
STRING | Characters that must appear; names not in the CSV still work but carry no tags |
copyright_pick |
COMBO | Work dropdown listing every work in the CSV (about 3,460 with the default data, sorted by popularity = the sum of the count column, descending); the first entry is (不限). Searchable as you type, case-insensitively |
describe_character |
BOOLEAN | Whether to append each character's fixed appearance tags (core_tags, i.e. the character description) to its line; on by default |
custom_csv_path |
STRING | Custom character CSV path; empty uses danbooru_character_001.csv |
tested_db_path |
STRING | Records database path; empty uses rollingcharacter.sqlite (separate from the artist node) |
CSV read limit (csv_max_rows) |
INT | Maximum number of CSV records consumed per read (includes the header row; 0 = unlimited, default 0). Records beyond the limit are not loaded, so the candidate pool, copyright menu and popularity pool shrink accordingly; set a positive value when the table is large and only the first part is needed |
This node performs no weight assignment (hence no weight parameters), no exhaustion (characters are single tags) and has no dedup mode parameter (repeats are avoided with a fixed-rule window, see below).
Work filter: a single dropdown
- The dropdown lists every work in the CSV (about 3,460 with the default data), sorted by
popularity: the
countcolumn summed over the work's characters, descending. That reflects "how popular the work is overall" instead of "how many characters it has"; ties are broken by name ascending, so the order is stable and reproducible - The first entry
(不限)means no filtering - It is searchable as you type, case-insensitively — typing
akeeps works starting with a, typingabnarrows it further - Both the raw
copyrightform (fate_(series)) and the escapedtriggerform (fate_\(series\)) are accepted - The options come from the default CSV (
danbooru_character_001.csv): pointingcustom_csv_pathat another file does not change them - If nothing is left after filtering the node errors out instead of silently emitting an empty prompt
Repeat avoidance: no repeat within the last 10 outputs
There is no dedup mode any more; instead a fixed-rule in-memory window is used:
- The window holds the last 10 output records, where one record = one generation (possibly several characters)
- Before sampling, the window's character names are removed from the candidates, so drawing 1 character ten
times in a row yields ten different characters; with
character_count=3it means "no repeat within the last 30 characters" - The window lives in the node instance's memory: restarting ComfyUI clears it, and several character nodes in one workflow each keep their own
- To clear it from a script, call
node.reset_recent();node.recent_names()shows the current window - When the window covers every candidate (for example when a very small work is selected), the node falls back to allowing repeats and logs a WARNING — it never errors out or gets stuck
Records database (SQLite)
- The default database lives in the node directory:
rollingcharacter.sqlite, separate from the artist node (records carry akind=charactermark, so they never impersonate artist records even in a shared file) - It only stores the output history (for
tested_countand later inspection); repeat avoidance does not depend on it, so changingtested_db_pathdoes not reset the window - The default database is cleared on every ComfyUI startup
Notes
- About 22% of characters have no character description (
core_tagsempty); those lines carry only the character and copyright tags - About 10 description tags per character on average, so lines get noticeably longer with many characters
- Character names are unique (no duplicates across 34,416 rows), so a name maps directly to its record
- The weight column in stored records is a
1.0placeholder (characters carry no weights); it only exists to keep record keys distinct