* feat(core): stream CSV records and add a configurable read limit
CSV reading used to build a full List[List[str]] of the raw file before
parsing, so cold loads kept both the raw rows and the final objects in
memory at once. Records now stream straight into the parsers.
- artists: _read() feeds _iter_csv_rows() into parse_artist_rows() instead
of _csv_rows(); a shared _iter_csv_rows() owns counting, early stop and
error containment for both read paths.
- characters: build_records() is rewritten as a single pass and accepts
Iterable. The previous Sequence signature iterated twice, so a generator
input silently dropped the first data row (6 rows -> 4 records, no error).
- characters: load_records()/load_index() read via the new
ArtistRepository.open_rows(), so the raw rows are no longer parked in
_rows_cache. load_rows() stays as the public compatibility entry.
- limits: normalize_max_rows()/resolve_max_rows() accept only None or a
positive integer. The effective limit is part of every cache key, and the
count is defined once for both paths: csv.reader records, header and
blank records included.
- exact-N no longer claims truncation: the streaming reader flags
"truncated" only when it actually reaches record N+1, with no extra read.
- nodes: both take an optional csv_max_rows (default 0 = unlimited),
appended last in optional and in the signature, marked advanced. ComfyUI
1.55.18 restores widgets_values positionally (NamedValuesRestore is off
by default), so appending at the end is what keeps old workflows intact.
Default behaviour is unchanged: same seeds produce byte-identical output
for both nodes.
* fix(core): tighten path boundaries and add administrator allowlists
Two boundary defects in ra_core/paths.py, plus an opt-in allowlist channel
for administrators who want to narrow the allowed roots.
- _resolve_unc_to_drive() used to map the admin-share form of ANY host to a
local drive: \\evil-host\D$\data\x.csv became D:\data\x.csv and was then
ALLOWED. The local path was what actually got opened, so the remote intent
was silently downgraded to a local read/write. Only localhost, 127.0.0.1
and [::1] are restored now; every other host stays UNC and is rejected by
the existing UNC branch.
- _is_within() compared os.path.commonpath() against the root without
normcase, so mixed-case nesting was judged outside. Under the existing
blacklist this was unreachable (forbidden_roots() only keeps existing
directories and realpath normalises their case), but an allowlist root that
does not exist yet was rejected. Both sides are normcased now.
- New administrator allowlist, read from ra_config.json in the node directory
(csv_allowed_roots / db_allowed_roots). No environment variable is read:
reading one would add a python_environment_manipulation finding to the
Registry scan, and the package is already flagged for the five existing
reads. The file is absent by default, which keeps the current blacklist
behaviour.
- The allowlist only narrows. Passing it still has to clear the UNC rejection
and the forbidden-root loop. It is checked after _canonicalize() and before
is_path_guard_disabled(), so "allowlist plus escape hatch" fails closed
instead of the escape hatch silently winning.
- Invalid configuration fails closed with an actionable message: bad JSON,
non-object top level, empty or missing lists, non-string roots, relative
roots and bare drive letters are all rejected.
- for_write is no longer a dead parameter: it selects db_allowed_roots over
csv_allowed_roots, so the two kinds stay separate.
- Node-directory data files are exempt: the bundled default CSV/DB are
trusted runtime resources, and users sometimes write the full default path
into custom_csv_path. Node source files are still blocked.
TOCTOU is unchanged and not claimed: check_db_path validates once and
sqlite3 reopens by path, and sqlite3 has no fd parameter.
* chore(release): prepare 5.2.0 docs and version bump
Documentation and version housekeeping for the 5.2.0 line. No runtime code
changes in this commit.
- CHANGELOG: new `## [未发布]` section covering the streaming CSV read, the
configurable read limit, the two path-boundary fixes, the administrator
allowlist and the publish-hygiene rule that had been missing. The section
is renamed to `## [5.2.0] - <date>` at release time, per the existing
discipline (every earlier release was made by renaming, not by adding).
- BENCHMARKS: same-machine old-vs-new comparison for the streaming rewrite
(artist cold peak 13.30 -> 3.39 MB, character cold peak 27.26 -> 16.46 MB,
character resident 26.86 -> 16.07 MB, character cold load 3129.7 ->
2703.0 ms) and the effect of the limit (artist cold load 1545 -> 32.6 ms
at max_rows=1000). Also indexes the behaviour-level regression tests that
guard these, none of which assert absolute timings.
- README: test-suite size corrected to the measured values (304 test
functions / 396 collected cases). The previous 254 / 338 had drifted.
- .gitignore: add *.egg-info/, build/, dist/, .pytest_cache/, .ruff_cache/
and .mypy_cache/. `pip install -e ".[dev]"` creates
ComfyUI_RollingArtist.egg-info/ in the repo root, which was not ignored,
so a local install dirtied `git status` and broke the "diff matches plan"
gate step.
- pyproject: version 5.1.7 -> 5.2.0.
Note: this branch is stacked on the two feature PRs; it only contains the
docs/version delta. Merging it before them would leave the changelog
describing code that is not on dev yet.
StarAsh042