Commit Graph
54 Commits
Author SHA1 Message Date
Dariusz L abb92b2138 refactor(frontend): clean unused catches and regex escapes 2026-08-02 04:20:44 +02:00
Dariusz L 4e0d11e7cc refactor(frontend): remove unused imports and locals 2026-08-02 04:14:54 +02:00
Dariusz L a8ba99800c Expand HuggingFace author fallback to multiple known publishers
Support both Comfy-Org and Kijai author indexes as known HuggingFace fallbacks, aggregate their status in the backend, refresh the cached author index, and update the frontend UI and tests.
2026-07-29 16:18:25 +02:00
Dariusz L 31ad1cde7c refactor: introduce Model Resolver-owned metadata sidecars and build modes 2026-07-27 15:00:18 +02:00
Dariusz L 16a69f6c47 refactor: unify provider JSON requests, catalog sync, and normalization across backend; add frontend HTML escaping helpers 2026-07-20 15:13:57 +02:00
Dariusz L 0047514f7c Add auto_open_on_missing setting to control resolver auto-launch 2026-07-19 15:21:04 +02:00
Dariusz L 79e7aa3743 refactor: unify provider JSON requests and move settings schema to dynamic loading 2026-07-18 15:10:34 +02:00
Dariusz L b302566542 refactor: remove dead code and add clear_completed_downloads endpoint 2026-07-18 12:57:29 +02:00
Dariusz L 2987c036e2 refactor: unify settings schema and dynamic node rules exposure across backend and frontend 2026-07-18 10:21:49 +02:00
Dariusz L be12494953 refactor: centralize shared utilities across backend and frontend
Backend:
- Move SHA256 normalization logic from path_utils._normalize_header_sha256
  into core/type_utils.normalize_sha256, adding support for 0x-prefixed
  hashes and unifying the normalization path used by
  extract_safetensors_header_sha256.
- Add tests for the 0x prefix normalization cases.

Frontend:
- Introduce copyTextWithFeedback in web/resolver/utils/html_utils.js
  as a shared clipboard helper with configurable feedback text, CSS class,
  and duration, replacing repeated inline clipboard copy logic in
  globals.js, model_info_methods.js, and the notification utility.
- Centralize showNotification on ResolverManagerDialog and remove the
  duplicate implementation from missingBrowserMethods.
- Replace local const localStorage = safeStorage aliases in
  model_info_methods.js, missing_browser_methods.js, and
  options_methods.js with direct safeStorage usage.
2026-07-17 18:06:22 +02:00
Dariusz L 8823049f9e refactor: centralize filename extraction, remote URL normalization, and storage abstractions 2026-07-16 10:13:38 +02:00
Dariusz L 54dae10be5 refactor: centralize background task handling and deduplicate title matching
- Add run_in_background_thread() in __init__.py to unify success/cancel/error
  handling for hash calculation and local metadata build progress
- Extract match_model_by_title_generic() in core/matcher.py and use it from
  both CivitAI and CivArchive providers to remove duplicated version/file
  selection logic
- Enable safetensors header SHA256 extraction by default and report hash source
  through progress updates
- Update options_methods.js to match backend progress behavior
- Add tests/test_title_matching_hashing.py covering title matching and hashing
- Minor metadata updates
2026-07-15 13:27:04 +02:00
Dariusz L 8cf688eba6 refactor: centralize source API requests, polling, and path parsing
- Add request_source_response and request_source_json helpers with 429 retry logic
- Introduce shared split_path_segments and parse_provider_model_url utilities
- Migrate CivitAI, CivArchive, HuggingFace, and ModelList sources to centralized helpers
- Add generic pollBackgroundTask utility for frontend progress polling
- Add tests for request_source_json, split_path_segments, parse_provider_model_url
- Add huggingface-hub to requirements
2026-07-14 12:03:35 +02:00
Dariusz L 48db08ae1a Harden network requests, downloads, and path handling
- Add core/network_utils.py with URL validation, public-IP enforcement,
  redirect validation, and safe request_public_url/validate_public_http_url helpers
- Replace direct requests calls in downloader/type_utils/sources with
  request_public_url so redirects are validated before follow-up
- Replace netloc string checks with host_matches_domain for CivitAI,
  CivArchive, and HuggingFace host matching
- Add HuggingFace Xet download backend with progress reporting and
  cancellation support via hf_xet/huggingface_hub
- Harden aria2 installer: reject symlinks/special files in archives,
  restrict custom aria2c paths to the managed install, and update UI copy
- Normalize paths with realpath and enforce configured model roots in
  local hash/metadata routes, plus remove caller-selected metadata_path
  injection in sidecar writing
- Sanitize download/network errors to strip signed query strings before
  logging or showing them to the user
2026-07-14 01:33:19 +02:00
Dariusz L c40cbafc00 Add Model Resolver Opener dependency marker node 2026-07-06 21:36:31 +02:00
Dariusz L f4a049eca9 feat(core, web): embed workflow model hash metadata and prefer hash-based matching 2026-07-06 18:09:31 +02:00
Dariusz L 0cecfab2a0 Switch default log level from DEBUG to ERROR across backend, frontend, and options defaults 2026-07-06 16:05:50 +02:00
Dariusz L 23c0f79444 Add paginated show-more behavior to metadata build history 2026-07-06 14:57:44 +02:00
Dariusz L 8f525044e8 feat(options): make metadata panels collapsible and add summary icons 2026-07-06 14:51:15 +02:00
Dariusz L 5a9e81cc70 refactor(options): use visibility-based transitions for metadata build panels and defer history rendering 2026-07-06 13:58:31 +02:00
Dariusz L f3c0024d46 Replace metadata build history select with custom action filter menu 2026-07-05 22:47:07 +02:00
Dariusz L 39943d5e17 Make metadata history filterable and full-length, and stabilize build progress UI 2026-07-05 22:25:01 +02:00
Dariusz L b9f364e3e4 Add Local Metadata Builder UI and backend API
This change introduces a complete Local Metadata Builder feature in the resolver
options panel. It allows users to scan local model files, create or update
.metadata.json sidecars, fill missing SHA256 hashes from safetensors headers or
file hashing, and review run history with cancel/error handling.

Changes include:
- backend API: /model_resolver/metadata-build/{capabilities,start,progress/{id},cancel}
- backend worker: core/metadata_builder.py with parallel metadata generation
- frontend section: Local Metadata Builder tab in Options
- frontend polling UI: progress, worker breakdown, history table, status/notifications
2026-07-05 21:52:57 +02:00
Dariusz L de6b7738e1 Reorganize options sidebar into Tools and System sections
Move Metadata Sizes into a dedicated Tools group and Maintenance into a System group to improve navigation clarity in the resolver options panel.
2026-07-05 20:59:14 +02:00
Dariusz L 85bbbef06d Add metadata size audit for model sidecar metadata 2026-07-05 18:03:29 +02:00
Dariusz L 4091dbb88d feat(civitai): add public API search fallback and improve search robustness 2026-07-03 22:00:13 +02:00
Dariusz L 28d054b228 refactor: deduplicate progress tracking, file size extraction, and loaded model rendering
- Replace inline analysis/hash/search progress helpers with direct JobProgressTracker calls
- Add shared extract_file_size() utility and remove duplicated extraction in CivitAI/CivArchive/HuggingFace
- Extract getModelNameAndStrength/getModelToken helpers in loaded models view
- Add raw response mode to fetchJson and simplify backend log export handling
2026-07-02 23:13:37 +02:00
Dariusz L 2d2e8b4621 refactor: centralize filename and size utilities across sources
- Introduce get_filename_from_path in path_utils and replace repeated
  os.path.basename calls across core/downloader.py, core/resolver.py,
  core/sources/civarchive.py, core/sources/civitai.py, and
  core/sources/huggingface.py
- Move duplicated inline HTTP response size helpers (_response_file_size,
  _parse_size_header, _parse_content_range_size) into core/type_utils.py
  and update civarchive.py and huggingface.py to import them
- Remove inline normalize_sha256_value from __init__.py and reuse the
  existing normalize_sha256 helper
- Refactor options_methods.js localStorage read/write logic into a
  SETTINGS_MAP-driven loop to reduce repetition
2026-07-02 03:37:44 +02:00
Dariusz L 59e61ba077 refactor: centralize progress reporters, default user-agent, improve error propagation and aria2 fetch usage 2026-07-02 02:54:38 +02:00
Dariusz L 2078d870b3 refactor: centralize path normalization and extract common options task handler
- Introduce normalizePathToForward/normalizePathToBackward helpers in render_format_methods.js
- Replace inline path normalization regex across 6 files with shared helpers
- Extract executeOptionsTask helper in options_methods.js to deduplicate repeated async loading/error/success handling and add missing error notifications
2026-07-02 02:12:40 +02:00
Dariusz L 1c71b609f3 fix: defer base model mapping refresh during batch load
Avoid redundant refreshBaseModelMappingOptions calls when loading saved
mappings by batching the refresh to a single call after all rows are added.
2026-07-01 23:19:02 +02:00
Dariusz L 22be853f13 Add FATAL log level to logging system 2026-07-01 20:43:32 +02:00
Dariusz L ef8ff03743 fix: remove forced download path mode reset on template detection 2026-06-30 23:16:44 +02:00
Dariusz L 25e20eea8f feat: add frontend and backend log export in options 2026-06-30 23:14:37 +02:00
Dariusz L 24db5ed54c refactor: centralize notification handling into shared utility 2026-06-30 01:08:11 +02:00
Dariusz L 54f861a02e feat: add aria2 daemon start/stop control and install migration
- Add /aria2/start and /aria2/stop API endpoints for explicit daemon lifecycle control
- Track aria2 daemon state (managed/running/version/active_transfers) in get_aria2_status
- Implement idle auto-stop timer (5 minutes) with aria2_auto_stop_daemon setting
- Add start_aria2_daemon() and stop_aria2_daemon() helpers in core/downloader.py
- Migrate legacy release-* install directories to normalized versioned paths
- Normalize aria2 version strings (strip release-/v prefixes) in installer metadata
- Update aria2 archive extraction to extract-to-temp then copy into canonical dir
- Add frontend daemon controls: status display, stop button, auto-stop toggle
2026-06-30 00:03:08 +02:00
Dariusz L 8b15500ce4 Add automatic aria2 binary installation support
- New core/aria2_installer.py module downloads the latest aria2c release
  from GitHub for Windows, macOS, and Linux with safe path extraction.
- New POST /model_resolver/aria2/install endpoint triggers installation
  and saves the resolved aria2c_path plus download_backend=aria2.
- Options UI adds an Install aria2 button, hides aria2-specific fields
  when another backend is selected, and updates status on completion.
- Includes .gitignore entry for tools/aria2/ so installed binaries are
  not committed.
2026-06-29 21:55:27 +02:00
Dariusz L aa91d1410c feat: add aria2 download backend with pause/resume support
Integrates aria2c as an alternative download backend alongside the built-in
Python downloader. Adds settings for selecting the backend and configuring
the aria2c executable path, plus a status check button in the options panel.

Backend changes:
- core/downloader.py: aria2 daemon lifecycle, JSON-RPC client, pause/resume
  state machine, and transfer tracking
- core/settings.py: download_backend normalization and aria2c_path setting
- __init__.py: new API routes for pause, resume, and aria2 status checks

Frontend changes:
- options_methods.js: backend selector, aria2c path input, check button,
  and local storage persistence
- resolve_download_methods.js: pause/resume handlers, pending status
  transitions, and progress rendering for aria2 downloads
- queue_methods.js: queue panel pause/resume buttons, backend badge, and
  debounced panel updates
- search_panel.js: actionsHtml support for composite action buttons
2026-06-27 23:39:31 +02:00
Dariusz L d49ae0becc refactor: centralize API calls through ResolverManagerDialog.fetchJson
Replace scattered api.fetchApi(...) + response.ok + response.json() boilerplate
across all web/modules with a unified this.fetchJson() helper defined on the
base dialog class.

Changes:
- dialog.fetchJson(url, options, context) handles headers, status checks, and
  JSON parsing in one place
- Extracts server-side error messages (errData.error) for better UX
- Optional silent flag suppresses notifications for background/refresh requests
- All callers simplified: async return, shorter bodies, consistent error handling
- BOM removed from workflow_update_methods.js and tabs_loaded_methods.js
2026-06-25 02:01:18 +02:00
Dariusz L dcac1b7c8a refactor: consolidate shared utilities and relocate view-specific settings logic
- Moved formatting helpers (formatBytes, formatDuration, getDownloadEtaText,
  formatDownloadProgressMeta) and getLocalMatchDisplayInfo from
  resolve_download_methods.js and missing_browser_methods.js into
  render_format_methods.js for shared reuse.
- Relocated settings-related methods (getStoredTokens,
  applyFrontendLoggingPreference, loadSettingsFromServer) and their
  log_system imports from download_target_methods.js into
  options_methods.js, where they are actually consumed by the options UI.
- Simplified missing_browser_methods.js to use
  getLocalMatchDisplayInfo instead of inline local-match code.
2026-06-24 23:42:34 +02:00
Dariusz L c5c3490210 feat: add auto-refresh of ComfyUI model catalog after Apply 2026-06-24 19:22:55 +02:00
Dariusz L 92be0d6213 Improve base model alias resolution and update model metadata
- Add version-trimmed alias generation to handle trailing version zeros (e.g. '4.0' -> '4')
- Track newly added base model names in 'new_models_added_list' field
- Include Ideogram 4.0, Boogu, and PolyGen in base-models registry
- Update web resolver to match aliases using both original and version-trimmed tokens
- Enhance base models update UI to show which new models were added
2026-06-21 02:29:42 +02:00
Dariusz L 47d286620f feat: add configurable download path system with templates and mappings
Implements a new download path configuration UI and backend logic:
- Add Subfolder fill mode: Suggested (existing), Template, Manual
- Add Base Model Path Mappings to override detected base-model folders
- Add Default Root settings to control root directory per category
- Add Detect From Existing Folders button that infers templates and mappings from local model folders
- Add backend /model_resolver/root-directories and /model_resolver/path-template-suggestions routes
- Refactor resolver settings into core/settings.py for shared frontend/backend use
- Extract path template inference logic into core/path_templates.py
- Update download panel, context menu, and loading flow to use new path settings
2026-06-19 00:09:58 +02:00
Dariusz L a4d5bd4c5c feat(logging): add frontend/backend log controls, unify console logging
- Add frontend_logs_enabled/frontend_log_level and backend_logs_enabled/backend_log_level
  toggles and selectors in the Options panel (web/resolver/views/options_methods.js)
- Sync frontend logging settings from localStorage / server settings into the
  frontend logger on startup and settings load (download_target_methods.js)
- Apply backend logging settings from server settings in the Python backend
  (__init__.py), loading control state on startup and after saves
- Make frontend Logger resilient to stored configs that lack an 'enabled' flag
  and to legacy/invalid moduleSettings levels (logger.js)
- Guard default module logger level initialization against undefined settings
  (log_funcs.js)
- Replace scattered console.log/usages with module-level log.debug calls across
  resolve_download_methods.js, model_resolver.js, and workflow_state_methods.js
- Bump log system package version to 1.5.6 (core + web)
2026-06-17 01:51:29 +02:00
Dariusz L 91a998bea2 feat: generalize download subfolder suggestions for all model types
- Replace CivitAI-specific subfolder suggestion with broadened logic that derives candidates from model metadata, filename patterns, and existing folder structure
- Add getSuggestedModelSubfolderCandidates / getSuggestedExistingSubfolderByModelName helpers
- Normalize download category consistently via normalize_download_category in backend and frontend
- Backend now walks actual ComfyUI folder paths dynamically instead of hardcoded category mapping
- Rename frontend methods to applySuggestedDownloadSubfolder / forceSuggestedDownloadSubfolder
- Update Auto-fill subfolder tooltip to reflect expanded scope across all model categories
2026-06-12 17:02:28 +02:00
Dariusz L 9160b365b0 feat: update nav icons for Local Database and Maintenance options 2026-06-07 18:18:07 +02:00
Dariusz L 5e52cab664 feat: add auto-fill subfolder toggle to control download target behavior 2026-06-07 18:10:40 +02:00
Dariusz L b7c4f47e90 feat: add auto-fill base model toggle to control default search behavior
Adds a new 'Auto-fill model filter' setting under Search Defaults that
controls whether new searches default to 'Auto' (detecting the workflow
base model) or 'none' (Any model). The preference is persisted in
localStorage as ModelResolver.autoFillBaseModel and is respected across
search panel initialization, base model dropdown defaults, workflow state
cloning, and dirty-state checks.
2026-06-07 18:02:42 +02:00
Dariusz L ab59cd231f feat: add base model filtering across search sources, API routes, and frontend UI 2026-06-06 19:43:43 +02:00
Dariusz L 57018b4e0f feat: add search result timestamps, enhanced cache clearing, and maintenance UI
Add searched_at timestamps to search results so cached entries are accurately tracked and preserved across merges. Expand the clear-search-cache command to also invalidate popular databases, model list, and the new result timestamp store. Frontend gains new cache-clearing methods and a full resolver cache reset for maintenance. The search panel and missing browser now show source result dates and toggle the search button between 'Search' and 'Search Again' when results already exist.
2026-06-06 16:38:30 +02:00