Support both Comfy-Org and Kijai author indexes as known HuggingFace fallbacks, aggregate their status in the backend, refresh the cached author index, and update the frontend UI and tests.
- Add core/network_utils.py with URL validation, public-IP enforcement,
redirect validation, and safe request_public_url/validate_public_http_url helpers
- Replace direct requests calls in downloader/type_utils/sources with
request_public_url so redirects are validated before follow-up
- Replace netloc string checks with host_matches_domain for CivitAI,
CivArchive, and HuggingFace host matching
- Add HuggingFace Xet download backend with progress reporting and
cancellation support via hf_xet/huggingface_hub
- Harden aria2 installer: reject symlinks/special files in archives,
restrict custom aria2c paths to the managed install, and update UI copy
- Normalize paths with realpath and enforce configured model roots in
local hash/metadata routes, plus remove caller-selected metadata_path
injection in sidecar writing
- Sanitize download/network errors to strip signed query strings before
logging or showing them to the user
Replace duplicated filename-to-query logic with a single
\uild_filename_search_queries\ helper in matcher/civarchive/huggingface,
standardize Civitai URL generation in the frontend via \getCivitaiModelUrl\,
unify browser user-agent and response file-size extraction in the downloader
path, and derive workflow node-type category hints from widget categories
rather than maintaining a large static mapping.
Replace duplicated SHA256 normalization, metadata sidecar discovery, trained-words
extraction, type coercion, and cache helpers into shared modules. Clarify that the
_civitai.py/_civarchive.py/_huggingface.py/_workflow_analyzer.py refactorings are
behavior-preserving so future contributors keep behavior aligned in one place.
- Move URL model-file validation logic to type_utils.looks_like_model_file and use it from __init__.py, civarchive
- Extract remote file-size probing into type_utils.fetch_remote_file_size and replace duplicated HEAD+Range logic in civarchive and huggingface
- Extract image normalization into type_utils.normalize_model_image, replacing duplicated CivitAI/CivArchive image mapping in civarchive and civitai
- Drop redundant local wrapper functions (_as_dict, _as_list, _version_sort_key, _select_primary_model_file, _extract_primary_file, _read_json_file, _write_json_file_atomic, _get_sha256_hash, _extract_civitai_image_id, _build_civitai_image_url) in favor of existing shared implementations
- Use path_utils.read_json_safe and write_json_atomic directly in model_list
- Introduce get_filename_from_path in path_utils and replace repeated
os.path.basename calls across core/downloader.py, core/resolver.py,
core/sources/civarchive.py, core/sources/civitai.py, and
core/sources/huggingface.py
- Move duplicated inline HTTP response size helpers (_response_file_size,
_parse_size_header, _parse_content_range_size) into core/type_utils.py
and update civarchive.py and huggingface.py to import them
- Remove inline normalize_sha256_value from __init__.py and reuse the
existing normalize_sha256 helper
- Refactor options_methods.js localStorage read/write logic into a
SETTINGS_MAP-driven loop to reduce repetition
Replace global log_* wrapper functions with create_module_logger(__name__)
instances across the codebase, enabling proper module-level logger
namespacing. Simplifies core/log_system/log_funcs.py by removing legacy
_normalize_module_path and cleaning up _normalize_module_name. Bumps
log_system config to 2.0.0 and introduces PROJECT_ALIASES for cleaner
log module prefix stripping.
- Add get_comfy_root_path() to core/path_utils.py and use it in __init__.py and core/downloader.py instead of inline logic
- Add select_primary_model_file(), parse_size_to_bytes(), get_version_sort_key(), and check_credential_http() to core/type_utils.py
- Replace duplicated credential-check helpers in civitai.py, huggingface.py with check_credential_http()
- Replace duplicated size parsing and version sorting in civarchive.py, civitai.py with shared utilities
- Replace duplicated primary-file selection in civarchive.py, civitai.py, lora_manager_archive.py with select_primary_model_file()
- Replace inline tempfile-based JSON writes in popular.py with write_json_atomic()
- Remove dead reload_model_list() from model_list.py
- Fix workflow_analyzer.py hint prioritization for diffusion_models vs specific output hints
- Add first_non_empty() and to_int() to core/type_utils.py
- Add get_filename_from_path() and METADATA_DIR to core/path_utils.py
- Replace inline implementations across resolver, downloader, and sources
with calls to the shared utilities
- Move strip_known_model_extension from resolver to matcher
Add core/type_utils.py with as_dict/as_list helpers, and
core/progress.py with a unified report_progress() callback.
Replace inline duplicated implementations of _json_safe_metadata(),
_as_list(), normalize_filename(), and per-source _report_progress()
with calls to the shared modules across downloader, matcher, resolver,
and all source modules (CivitAI, CivArchive, HuggingFace, LoRA Manager).
Extend filename normalization to include dots as separators.
Update search logging to use reusable formatters for values, sizes, and field
pairs, and add append-only extra kwargs so downstream logs stay on one line.
Tighten HuggingFace source messages, refresh the author index metadata, and
consolidate LoraManager/CivitAI log text.
Adds a checkbox in the resolver footer search menu that bypasses all
search result caches for the duration of a batch search run.
Frontend (queue_methods.js, resolve_download_methods.js,
resolver_dialog.js, search_panel.js, resolver-main.css):
- Introduce checkbox-type footer menu items (type: 'checkbox').
- Add 'Force search (ignore cache)' checkbox to the search menu,
wired to a new forceBatchSearch flag on ResolverManagerDialog.
- Pass forceSearch through searchMissingBatch and searchOnline to
the backend API call.
- When the manual search button is clicked and results already exist
for that model, forceSearch defaults to true.
- mergeSearchResults respects forceRefresh: if a forced search
returns no results for a source, that source is cleared from the
cached results.
Backend (__init__.py, core/sources/huggingface.py):
- force_search is read from the incoming API request and, when true,
clears all active source search caches (local, huggingface,
civitai, civarchive, lora_manager_archive) before searching.
- force_search is forwarded as force_refresh to
search_huggingface_for_file, which folds it into the cache key so
the cache is not hit, and propagates it to _get_author_index.
- Search result timestamps are no longer inherited from prior
cached signatures when force_search is active; a fresh timestamp
is written unconditionally.
Updated metadata/huggingface-author-index.json reflects the latest
cached author index after rebuild.
Add searched_at timestamps to search results so cached entries are accurately tracked and preserved across merges. Expand the clear-search-cache command to also invalidate popular databases, model list, and the new result timestamp store. Frontend gains new cache-clearing methods and a full resolver cache reset for maintenance. The search panel and missing browser now show source result dates and toggle the search button between 'Search' and 'Search Again' when results already exist.
Add support for an optional CivitAI session token and cache-management for search results. UI: new input for __Secure-civitai-token saved to localStorage (modelLinker.civitaiSessionToken) and cleared via the options dialog; saving/clearing tokens triggers a backend cache clear and updates notifications. Backend: add clear_search_cache() helpers in core.sources.huggingface and core.sources.civitai, register /model_linker/clear-search-cache route to flush both caches, and wire the civitai session token into search flows. CivitAI changes include sending the session token as a Cookie header for tRPC searches, incorporating session and model_type into the search cache key, and passing session_token through search_civitai_for_file and related calls. Purpose: ensure authenticated/session-aware search results (including NSFW/visibility differences) and avoid stale cached search results after token/settings changes.
Add extensive observability/logging across ModelLinker and source search modules. __init__.py: introduce summarize_result and log_search_result helpers, replace ad-hoc logger calls with log_info/log_exception, and add start/result/summary logs for local, popular, model_list, HuggingFace and CivitAI searches (including extra metadata like confidence, files_count, results_count). core/sources/civitai.py: log cache hits, API call start, response counts, model/version/file checks, no-result info and improved exception logging. core/sources/huggingface.py: log cache hits, API call start, response counts, per-repo debug info, repo tree sizes, no-result info and improved exception logging. Changes are focused on diagnostics and do not alter core search logic.
Replace direct uses of the logging module and module-level loggers with the centralized log_system.log_funcs (log_debug/log_info/log_warn/log_error/log_exception) across multiple modules. Updated imports and removed unused logging.getLogger() instances in downloader, scanner, sources (civitai, huggingface, model_list, popular), workflow_analyzer, workflow_updater, and __init__.py. Minor message/text tweaks and import formatting applied; behavior unchanged aside from routing logs through the shared logging helpers.
Introduce a new logging subsystem under core/log_system (logger.py, config.py, log_funcs.py) implementing AzLogs (LogLevel enum, colored console output, optional file rotation) and a lightweight wrapper (log_debug/info/warn/error/exception) with safe fallback when AzLogs is unavailable. Integrate these log functions across many core modules (downloader, linker, matcher, scanner, workflow_analyzer, workflow_updater, sources and related files) and export logger and helper log functions from core.__init__. Replace several direct logging calls in linker with the new log_* helpers and update core.__init__ exports. Also include small code style and robustness tweaks (path/filename normalization, JSON/file handling, minor refactors and formatting fixes) to improve matching and model-list/popular lookups.
Add support for the gguf model format across the codebase and improve handling of nested/dict-type widget model references. Changes include:
- Add ".gguf" to model extension lists in core/linker.py, core/scanner.py, core/sources/huggingface.py, and core/workflow_analyzer.py so gguf files are detected and suggested.
- Update HuggingFace search/repo listing to consider gguf files and improve some request/response handling and logging.
- Improve scanner implementation: formatting/PEP8 cleanup, robust extension checks, normalized MODEL_EXTENSIONS set, safer path handling, and clearer logging.
- Add NESTED_MODEL_KEYS and logic in workflow_analyzer to detect model filenames stored inside dict-type widget values (e.g. rgthree Power Lora Loader) and record a nested_key for resolution.
- Propagate nested_key through apply_resolution mapping and update update_model_path to write resolved paths back into nested dict keys when present (instead of only replacing the top-level widget value).
- Update web UI (web/linker.js) regexes to strip/display gguf extensions in model name formatting.
These changes allow gguf models to be discovered, resolved and updated in workflows, and enable support for nodes that store model references inside nested dict fields.
Dariusz L