Commit Graph
201 Commits
Author SHA1 Message Date
Dariusz L 1ee4c6ce4c workflow_analyzer: handle implicit control_after_generate widgets in dynamic layouts
Treat control_after_generate widgets that are implicitly inserted after seed
widgets as non-model widgets when analyzing dynamic node layouts. This
prevents the resolver from attempting to resolve control values such as
'fixed', 'randomize', 'increment', and 'decrement' as model paths.
2026-07-24 15:22:55 +02:00
Dariusz L ac9bf820c8 Handle control_after_generate widget indexes in workflow analyzer 2026-07-24 15:10:00 +02:00
Dariusz L 03c545e297 Improve workflow analyzer widget category hints and test coverage 2026-07-24 14:43:18 +02:00
Dariusz L 45ea998068 Resolve GGUF models by normalized download category alias 2026-07-24 14:11:25 +02:00
Dariusz L da7d9b3fc9 Skip non-model reference categories in workflow analyzer
Treat widget values under the configs and custom_nodes dynamic
categories as non-model references so the workflow analyzer does
not falsely mark them as missing models. Added a regression test
to verify a checkpoint widget is still detected when a config
widget is present.
2026-07-24 14:04:07 +02:00
Dariusz L d293b0aaf8 Treat model_gguf as diffusion_models and render accepted file formats in missing-model UI 2026-07-20 16:54:28 +02:00
Dariusz L 097be623f4 feat: preserve CivArchive page URLs separate from HuggingFace mirrors, fix HTML array escaping, and accept provider array payloads
Backend:
- Keep CivArchive source page URL distinct from HuggingFace download mirrors in download metadata
- Handle CivitAI hash lookup HTTP errors and unexpected JSON shapes without caching failures
- Allow network JSON requests to return arrays as well as objects

Frontend:
- Escape each item individually when interpolating arrays into HTML templates
- Prefer the selected provider's page URL over cross-provider mirrors in context menus and download targets
- Reuse source link resolution for model info display

Tests:
- Add coverage for CivArchive metadata URL isolation, HTML array escaping, provider array responses, and CivitAI hash lookup caching behavior
2026-07-20 16:16:09 +02:00
Dariusz L 60ca3a381a fix: isolate manual/HuggingFace metadata from stale CivitAI provenance 2026-07-20 15:34:32 +02:00
Dariusz L 16a69f6c47 refactor: unify provider JSON requests, catalog sync, and normalization across backend; add frontend HTML escaping helpers 2026-07-20 15:13:57 +02:00
Dariusz L 7ee358b2c5 Improve model matching by normalizing technical suffixes and family signatures 2026-07-20 14:15:41 +02:00
Dariusz L a4460afa65 fix(workflow_analyzer): ignore generic model_name STRING widgets in metadata/search nodes
Generic widget names such as model_name are no longer treated as local
model references by default. Only folder-backed schemas, known widget
indexes, or a model-typed combo/output pairing now admit an extensionless
value.

This prevents metadata/search nodes (for example Civitai Hash Fetcher)
with a model_name STRING input from being incorrectly reported as missing
models.

Added test_model_name_string_used_as_metadata_query_is_ignored to cover
the regression.
2026-07-19 19:34:10 +02:00
Dariusz L fb600a3868 Refine workflow model reference detection
- Accept explicit model URNs without requiring model widget metadata
- Restrict model scanning to widgets with category/model hints or model-like filenames
- Remove over-broad single-widget-value detection path
- Add tests for cached model text, custom selectors, and constant/model slot edge cases
2026-07-19 19:10:54 +02:00
Dariusz L 8b2d200f77 Normalize select_safetensors to diffusion_models 2026-07-19 15:46:34 +02:00
Dariusz L 0047514f7c Add auto_open_on_missing setting to control resolver auto-launch 2026-07-19 15:21:04 +02:00
HarrievG db2c77290c fix: support Hugging Face Hub 1.23+ Xet downloads
Handle removal of the legacy Xet connection-info helper while preserving the native download fallback. Adds regression coverage for mixed Hugging Face dependency versions.
2026-07-19 14:45:49 +02:00
Dariusz L 79e7aa3743 refactor: unify provider JSON requests and move settings schema to dynamic loading 2026-07-18 15:10:34 +02:00
Dariusz L 8cc02010b0 Remove unused backend and frontend code 2026-07-18 14:47:05 +02:00
Dariusz L 5362cb17a3 refactor: remove dead code and add isAutoOpenEnabled helper 2026-07-18 14:01:05 +02:00
Dariusz L 89a721fc62 Normalize CivitAI search queries and simplify tRPC search behavior 2026-07-18 13:43:51 +02:00
Dariusz L c39eea4f51 refactor: remove dead code and unused frontend handlers across backend and web 2026-07-18 13:30:34 +02:00
Dariusz L b302566542 refactor: remove dead code and add clear_completed_downloads endpoint 2026-07-18 12:57:29 +02:00
Dariusz L 0b53fca8c6 refactor: unify category resolution, network requests, sidecar lookup, and progress tracking
Consolidate provider-specific category mapping into resolve_model_category(),
shared JSON request handling into execute_provider_json_request(), sidecar
path resolution into get_or_build_model_sidecar_info(), and progress
reporting into ProgressTracker. Update civitai/civarchive/common sources to
use the unified helpers. Add tests for the new utilities. Remove redundant
frontend wrapper methods from render_format_methods.js.
2026-07-18 12:48:11 +02:00
Dariusz L 759aef99f1 Unify model result building and progress payload updates 2026-07-18 12:26:17 +02:00
Dariusz L 2987c036e2 refactor: unify settings schema and dynamic node rules exposure across backend and frontend 2026-07-18 10:21:49 +02:00
Dariusz L aff97c5e29 refactor: centralize shared normalization, timestamp, and search-result helpers 2026-07-18 10:02:34 +02:00
Dariusz L 719b3b3f43 Refactor: centralize metadata sidecar helpers and custom-result builders 2026-07-18 09:43:27 +02:00
Dariusz L be12494953 refactor: centralize shared utilities across backend and frontend
Backend:
- Move SHA256 normalization logic from path_utils._normalize_header_sha256
  into core/type_utils.normalize_sha256, adding support for 0x-prefixed
  hashes and unifying the normalization path used by
  extract_safetensors_header_sha256.
- Add tests for the 0x prefix normalization cases.

Frontend:
- Introduce copyTextWithFeedback in web/resolver/utils/html_utils.js
  as a shared clipboard helper with configurable feedback text, CSS class,
  and duration, replacing repeated inline clipboard copy logic in
  globals.js, model_info_methods.js, and the notification utility.
- Centralize showNotification on ResolverManagerDialog and remove the
  duplicate implementation from missingBrowserMethods.
- Replace local const localStorage = safeStorage aliases in
  model_info_methods.js, missing_browser_methods.js, and
  options_methods.js with direct safeStorage usage.
2026-07-17 18:06:22 +02:00
Dariusz L 20a83c7262 refactor: improve code quality and consistency across backend modules 2026-07-16 11:00:45 +02:00
Dariusz L 38dc3e4474 refactor: centralize shared utilities, normalize imports, and configure project linting 2026-07-16 10:52:06 +02:00
Dariusz L 04c3de9533 refactor: reuse shared path and URL normalization utilities
Replace inline regex path/URL normalization in resolver.py and
missing_browser_methods.js with the existing shared helpers:

- core/resolver.py: normalize_workflow_download_url now uses
  prepare_remote_size_probe_url from type_utils instead of a local
  huggingface-specific regex substitution.
- web/resolver/views/missing_browser_methods.js: path identity
  generation now uses normalizePathIdentity from html_utils.js rather
  than ad hoc replace/trim/lowercase logic.

This keeps normalization behavior consistent across the codebase and
removes duplicated regex patterns.
2026-07-16 10:27:06 +02:00
Dariusz L 8823049f9e refactor: centralize filename extraction, remote URL normalization, and storage abstractions 2026-07-16 10:13:38 +02:00
Dariusz L 32394135e9 refactor: centralize catalog management and unify matcher/progress logic 2026-07-16 09:51:47 +02:00
Dariusz L 54dae10be5 refactor: centralize background task handling and deduplicate title matching
- Add run_in_background_thread() in __init__.py to unify success/cancel/error
  handling for hash calculation and local metadata build progress
- Extract match_model_by_title_generic() in core/matcher.py and use it from
  both CivitAI and CivArchive providers to remove duplicated version/file
  selection logic
- Enable safetensors header SHA256 extraction by default and report hash source
  through progress updates
- Update options_methods.js to match backend progress behavior
- Add tests/test_title_matching_hashing.py covering title matching and hashing
- Minor metadata updates
2026-07-15 13:27:04 +02:00
Dariusz L 7e724d373f refactor: complete migration to centralized HTTP helpers and shared normalization
- Replace remaining direct requests.get calls with request_public_url,
  request_source_response, and request_source_json in downloader,
  civarchive, civitai, and popular sources
- Use normalize_model_file_info in CivitAI source instead of inline logic
- Update huggingface-author-index.json metadata
- Add tests for civarchive page/search requests and popular base model updates
- Adjust downloader redirect test for new request_public_url behavior
2026-07-15 13:04:00 +02:00
Dariusz L 8cf688eba6 refactor: centralize source API requests, polling, and path parsing
- Add request_source_response and request_source_json helpers with 429 retry logic
- Introduce shared split_path_segments and parse_provider_model_url utilities
- Migrate CivitAI, CivArchive, HuggingFace, and ModelList sources to centralized helpers
- Add generic pollBackgroundTask utility for frontend progress polling
- Add tests for request_source_json, split_path_segments, parse_provider_model_url
- Add huggingface-hub to requirements
2026-07-14 12:03:35 +02:00
Dariusz L 280ca2e6de Improve HuggingFace Xet downloads with native progress snapshots, transfer metrics, cancellation, and finalizing UI state 2026-07-14 11:02:57 +02:00
Dariusz L 1f0edaaa7e Harden aria2 downloads, retry RPC, and preserve HuggingFace XET redirect headers 2026-07-14 02:33:44 +02:00
Dariusz L 48db08ae1a Harden network requests, downloads, and path handling
- Add core/network_utils.py with URL validation, public-IP enforcement,
  redirect validation, and safe request_public_url/validate_public_http_url helpers
- Replace direct requests calls in downloader/type_utils/sources with
  request_public_url so redirects are validated before follow-up
- Replace netloc string checks with host_matches_domain for CivitAI,
  CivArchive, and HuggingFace host matching
- Add HuggingFace Xet download backend with progress reporting and
  cancellation support via hf_xet/huggingface_hub
- Harden aria2 installer: reject symlinks/special files in archives,
  restrict custom aria2c paths to the managed install, and update UI copy
- Normalize paths with realpath and enforce configured model roots in
  local hash/metadata routes, plus remove caller-selected metadata_path
  injection in sidecar writing
- Sanitize download/network errors to strip signed query strings before
  logging or showing them to the user
2026-07-14 01:33:19 +02:00
Dariusz L 7feea3b61e Restrict scanner extension filtering to configured category extensions only 2026-07-14 00:09:59 +02:00
Dariusz L 64fdb7bd87 Import build_search_result in civitai source module 2026-07-12 21:20:46 +02:00
Dariusz L 8cd3a2968b Use indented JSON when writing HuggingFace author index cache 2026-07-12 21:04:04 +02:00
Dariusz L 6451d9bcf4 Improve model path matching and refresh HuggingFace author index
core/path_templates.py: allow model alias tokens to match when suffixed
with a version string or plural form, reducing false negatives in
single-folder path resolution.

metadata/huggingface-author-index.json: update Comfy-Org index timestamp
and repository ordering.
2026-07-12 21:00:18 +02:00
Dariusz L c40cbafc00 Add Model Resolver Opener dependency marker node 2026-07-06 21:36:31 +02:00
Dariusz L f4a049eca9 feat(core, web): embed workflow model hash metadata and prefer hash-based matching 2026-07-06 18:09:31 +02:00
Dariusz L 0cecfab2a0 Switch default log level from DEBUG to ERROR across backend, frontend, and options defaults 2026-07-06 16:05:50 +02:00
Dariusz L b27572d84f Suppress noisy no-op metadata build log when all models were skipped 2026-07-06 15:23:54 +02:00
Dariusz L 39943d5e17 Make metadata history filterable and full-length, and stabilize build progress UI 2026-07-05 22:25:01 +02:00
Dariusz L b9f364e3e4 Add Local Metadata Builder UI and backend API
This change introduces a complete Local Metadata Builder feature in the resolver
options panel. It allows users to scan local model files, create or update
.metadata.json sidecars, fill missing SHA256 hashes from safetensors headers or
file hashing, and review run history with cancel/error handling.

Changes include:
- backend API: /model_resolver/metadata-build/{capabilities,start,progress/{id},cancel}
- backend worker: core/metadata_builder.py with parallel metadata generation
- frontend section: Local Metadata Builder tab in Options
- frontend polling UI: progress, worker breakdown, history table, status/notifications
2026-07-05 21:52:57 +02:00
Dariusz L 85bbbef06d Add metadata size audit for model sidecar metadata 2026-07-05 18:03:29 +02:00
Dariusz L 6c1e940203 refactor(sources): centralize search result construction and shared helpers
Move MODEL_TITLE_MATCH_THRESHOLD to core/matcher.py.

Add shared helpers to core/type_utils.py:
- normalize_lora_manager_type
- normalize_model_file_info
- build_search_result

Refactor civarchive, civitai, lora_manager_archive, and huggingface
sources to use centralized helpers, removing duplicated inline dict
construction, type normalization, username extraction, and search
query wrappers.
2026-07-05 06:34:16 +02:00