Commit Graph
532 Commits
Author SHA1 Message Date
Dariusz L be12494953 refactor: centralize shared utilities across backend and frontend
Backend:
- Move SHA256 normalization logic from path_utils._normalize_header_sha256
  into core/type_utils.normalize_sha256, adding support for 0x-prefixed
  hashes and unifying the normalization path used by
  extract_safetensors_header_sha256.
- Add tests for the 0x prefix normalization cases.

Frontend:
- Introduce copyTextWithFeedback in web/resolver/utils/html_utils.js
  as a shared clipboard helper with configurable feedback text, CSS class,
  and duration, replacing repeated inline clipboard copy logic in
  globals.js, model_info_methods.js, and the notification utility.
- Centralize showNotification on ResolverManagerDialog and remove the
  duplicate implementation from missingBrowserMethods.
- Replace local const localStorage = safeStorage aliases in
  model_info_methods.js, missing_browser_methods.js, and
  options_methods.js with direct safeStorage usage.
2026-07-17 18:06:22 +02:00
Dariusz L 28d24214aa fix: improve resolver error propagation, sanitization, and consistency 2026-07-16 11:11:48 +02:00
Dariusz L 20a83c7262 refactor: improve code quality and consistency across backend modules 2026-07-16 11:00:45 +02:00
Dariusz L 61f1403543 Update .gitignore 2026-07-16 10:53:08 +02:00
Dariusz L 38dc3e4474 refactor: centralize shared utilities, normalize imports, and configure project linting 2026-07-16 10:52:06 +02:00
Dariusz L 04c3de9533 refactor: reuse shared path and URL normalization utilities
Replace inline regex path/URL normalization in resolver.py and
missing_browser_methods.js with the existing shared helpers:

- core/resolver.py: normalize_workflow_download_url now uses
  prepare_remote_size_probe_url from type_utils instead of a local
  huggingface-specific regex substitution.
- web/resolver/views/missing_browser_methods.js: path identity
  generation now uses normalizePathIdentity from html_utils.js rather
  than ad hoc replace/trim/lowercase logic.

This keeps normalization behavior consistent across the codebase and
removes duplicated regex patterns.
2026-07-16 10:27:06 +02:00
Dariusz L 52a8d3a54b refactor: use get_filename_from_path helper for consistent filename extraction 2026-07-16 10:18:15 +02:00
Dariusz L 8823049f9e refactor: centralize filename extraction, remote URL normalization, and storage abstractions 2026-07-16 10:13:38 +02:00
Dariusz L 32394135e9 refactor: centralize catalog management and unify matcher/progress logic 2026-07-16 09:51:47 +02:00
Dariusz L e4485f3bdf Treat SAM and Ultralytics models as base-model-independent in search panel 2026-07-16 09:10:00 +02:00
Dariusz L 54dae10be5 refactor: centralize background task handling and deduplicate title matching
- Add run_in_background_thread() in __init__.py to unify success/cancel/error
  handling for hash calculation and local metadata build progress
- Extract match_model_by_title_generic() in core/matcher.py and use it from
  both CivitAI and CivArchive providers to remove duplicated version/file
  selection logic
- Enable safetensors header SHA256 extraction by default and report hash source
  through progress updates
- Update options_methods.js to match backend progress behavior
- Add tests/test_title_matching_hashing.py covering title matching and hashing
- Minor metadata updates
2026-07-15 13:27:04 +02:00
Dariusz L 7e724d373f refactor: complete migration to centralized HTTP helpers and shared normalization
- Replace remaining direct requests.get calls with request_public_url,
  request_source_response, and request_source_json in downloader,
  civarchive, civitai, and popular sources
- Use normalize_model_file_info in CivitAI source instead of inline logic
- Update huggingface-author-index.json metadata
- Add tests for civarchive page/search requests and popular base model updates
- Adjust downloader redirect test for new request_public_url behavior
2026-07-15 13:04:00 +02:00
Dariusz L a5e08653af Update README.md 2026-07-14 20:39:28 +02:00
Dariusz L 74d4a415bc Update README.md 2026-07-14 20:35:09 +02:00
Dariusz L 027d44e244 Update README.md 2026-07-14 20:30:28 +02:00
Dariusz L f3221282a3 Update huggingface-author-index.json 2026-07-14 12:12:35 +02:00
Dariusz L 8cf688eba6 refactor: centralize source API requests, polling, and path parsing
- Add request_source_response and request_source_json helpers with 429 retry logic
- Introduce shared split_path_segments and parse_provider_model_url utilities
- Migrate CivitAI, CivArchive, HuggingFace, and ModelList sources to centralized helpers
- Add generic pollBackgroundTask utility for frontend progress polling
- Add tests for request_source_json, split_path_segments, parse_provider_model_url
- Add huggingface-hub to requirements
2026-07-14 12:03:35 +02:00
Dariusz L a5b54bb7de Update pyproject.toml v1.0.4 2026-07-14 11:17:17 +02:00
Dariusz L cb1042b00d Add tests directory and remove tests/ from .gitignore 2026-07-14 11:15:51 +02:00
Dariusz L 280ca2e6de Improve HuggingFace Xet downloads with native progress snapshots, transfer metrics, cancellation, and finalizing UI state 2026-07-14 11:02:57 +02:00
Dariusz L 1f0edaaa7e Harden aria2 downloads, retry RPC, and preserve HuggingFace XET redirect headers 2026-07-14 02:33:44 +02:00
Dariusz L 48db08ae1a Harden network requests, downloads, and path handling
- Add core/network_utils.py with URL validation, public-IP enforcement,
  redirect validation, and safe request_public_url/validate_public_http_url helpers
- Replace direct requests calls in downloader/type_utils/sources with
  request_public_url so redirects are validated before follow-up
- Replace netloc string checks with host_matches_domain for CivitAI,
  CivArchive, and HuggingFace host matching
- Add HuggingFace Xet download backend with progress reporting and
  cancellation support via hf_xet/huggingface_hub
- Harden aria2 installer: reject symlinks/special files in archives,
  restrict custom aria2c paths to the managed install, and update UI copy
- Normalize paths with realpath and enforce configured model roots in
  local hash/metadata routes, plus remove caller-selected metadata_path
  injection in sidecar writing
- Sanitize download/network errors to strip signed query strings before
  logging or showing them to the user
2026-07-14 01:33:19 +02:00
Dariusz L 208bf06f3d Update README.md 2026-07-14 00:17:19 +02:00
Dariusz L 7feea3b61e Restrict scanner extension filtering to configured category extensions only 2026-07-14 00:09:59 +02:00
Dariusz L b0661b7d81 Update huggingface-author-index.json 2026-07-13 23:29:34 +02:00
Dariusz L 8ae6efcc82 Update pyproject.toml v1.0.3 2026-07-12 23:42:34 +02:00
Dariusz L 44153d9d7a Make download cancellation workflow-scoped and prevent stale progress from reviving cancelled downloads
- Introduce workflow-scoped download state keys so active downloads and progress are isolated per workflow tab/route.
- Add a transient 'cancelling' status path and a finalizeCancelledDownloadFrontend helper to avoid flicker and prevent stale progress responses from recreating completed/cancelled UI state.
- Guard active-download lookups by current workflow scope to reduce cross-workflow interference.
2026-07-12 23:28:42 +02:00
Dariusz L 6ed34b6a92 Preserve active download local matches and clean up cancelled downloads
Keep path-specific Local Matches for active downloads during refreshes,
URN search, and missing-model display so late hash-only lookups do not
overwrite them.

Remove cancelled download matches from missing-model UI and cached
analysis when a download is cancelled.
2026-07-12 22:04:53 +02:00
Dariusz L 64fdb7bd87 Import build_search_result in civitai source module 2026-07-12 21:20:46 +02:00
Dariusz L 572b948b92 Add Tripo and Hunyuan3D base models and refresh HuggingFace author index cache 2026-07-12 21:10:12 +02:00
Dariusz L 8cd3a2968b Use indented JSON when writing HuggingFace author index cache 2026-07-12 21:04:04 +02:00
Dariusz L 6451d9bcf4 Improve model path matching and refresh HuggingFace author index
core/path_templates.py: allow model alias tokens to match when suffixed
with a version string or plural form, reducing false negatives in
single-folder path resolution.

metadata/huggingface-author-index.json: update Comfy-Org index timestamp
and repository ordering.
2026-07-12 21:00:18 +02:00
Dariusz L 8baf3871d0 Update ComfyUIdownloads.yml 2026-07-07 18:49:15 +02:00
Dariusz L e0997ed9fd Refactor workflow dependency marker sizing to use fixed bounds and compact oversized nodes 2026-07-07 02:40:55 +02:00
Dariusz L 6f9aa9cc55 Improve Model Resolver dependency marker node compatibility and rendering
- Adopt ComfyUI's ComfyExtension/comfy_entrypoint API in __init__.py when
  available, falling back to legacy NODE_CLASS_MAPPINGS/DISPLAY_NAME_MAPPINGS
  on older runtimes so the node loads across ComfyUI versions.
- Hook into beforeRegisterNodeDef, loadedGraphNode, and afterConfigureGraph
  in web/resolver.js so marker nodes are patched during registration, graph
  load, and initial render.
- Teach model_resolver.js to recognise the marker node by every known class
  property shape, patch its type on creation, enforce a minimum size with
  setSize() when present, and dirty the canvas so changes are reflected.
2026-07-06 21:59:30 +02:00
Dariusz L c40cbafc00 Add Model Resolver Opener dependency marker node 2026-07-06 21:36:31 +02:00
Dariusz L f4a049eca9 feat(core, web): embed workflow model hash metadata and prefer hash-based matching 2026-07-06 18:09:31 +02:00
Dariusz L 5d5aa6ae76 Update pyproject.toml v1.0.2 2026-07-06 16:18:52 +02:00
Dariusz L 7962ab2050 Update publish.yml 2026-07-06 16:12:03 +02:00
Dariusz L 488f9512c8 Update pyproject.toml 2026-07-06 16:08:18 +02:00
Dariusz L 0cecfab2a0 Switch default log level from DEBUG to ERROR across backend, frontend, and options defaults 2026-07-06 16:05:50 +02:00
Dariusz L b27572d84f Suppress noisy no-op metadata build log when all models were skipped 2026-07-06 15:23:54 +02:00
Dariusz L 23c0f79444 Add paginated show-more behavior to metadata build history 2026-07-06 14:57:44 +02:00
Dariusz L 8f525044e8 feat(options): make metadata panels collapsible and add summary icons 2026-07-06 14:51:15 +02:00
Dariusz L 5a9e81cc70 refactor(options): use visibility-based transitions for metadata build panels and defer history rendering 2026-07-06 13:58:31 +02:00
Dariusz L f3c0024d46 Replace metadata build history select with custom action filter menu 2026-07-05 22:47:07 +02:00
Dariusz L 39943d5e17 Make metadata history filterable and full-length, and stabilize build progress UI 2026-07-05 22:25:01 +02:00
Dariusz L b9f364e3e4 Add Local Metadata Builder UI and backend API
This change introduces a complete Local Metadata Builder feature in the resolver
options panel. It allows users to scan local model files, create or update
.metadata.json sidecars, fill missing SHA256 hashes from safetensors headers or
file hashing, and review run history with cancel/error handling.

Changes include:
- backend API: /model_resolver/metadata-build/{capabilities,start,progress/{id},cancel}
- backend worker: core/metadata_builder.py with parallel metadata generation
- frontend section: Local Metadata Builder tab in Options
- frontend polling UI: progress, worker breakdown, history table, status/notifications
2026-07-05 21:52:57 +02:00
Dariusz L de6b7738e1 Reorganize options sidebar into Tools and System sections
Move Metadata Sizes into a dedicated Tools group and Maintenance into a System group to improve navigation clarity in the resolver options panel.
2026-07-05 20:59:14 +02:00
Dariusz L 85bbbef06d Add metadata size audit for model sidecar metadata 2026-07-05 18:03:29 +02:00