Commit Graph
87 Commits
Author SHA1 Message Date
Dariusz L 25e20eea8f feat: add frontend and backend log export in options 2026-06-30 23:14:37 +02:00
Dariusz L a7f5f70e3c refactor: migrate logging from global functions to per-module loggers
Replace global log_* wrapper functions with create_module_logger(__name__)
instances across the codebase, enabling proper module-level logger
namespacing. Simplifies core/log_system/log_funcs.py by removing legacy
_normalize_module_path and cleaning up _normalize_module_name. Bumps
log_system config to 2.0.0 and introduces PROJECT_ALIASES for cleaner
log module prefix stripping.
2026-06-30 21:24:33 +02:00
Dariusz L 54f861a02e feat: add aria2 daemon start/stop control and install migration
- Add /aria2/start and /aria2/stop API endpoints for explicit daemon lifecycle control
- Track aria2 daemon state (managed/running/version/active_transfers) in get_aria2_status
- Implement idle auto-stop timer (5 minutes) with aria2_auto_stop_daemon setting
- Add start_aria2_daemon() and stop_aria2_daemon() helpers in core/downloader.py
- Migrate legacy release-* install directories to normalized versioned paths
- Normalize aria2 version strings (strip release-/v prefixes) in installer metadata
- Update aria2 archive extraction to extract-to-temp then copy into canonical dir
- Add frontend daemon controls: status display, stop button, auto-stop toggle
2026-06-30 00:03:08 +02:00
Dariusz L 8b15500ce4 Add automatic aria2 binary installation support
- New core/aria2_installer.py module downloads the latest aria2c release
  from GitHub for Windows, macOS, and Linux with safe path extraction.
- New POST /model_resolver/aria2/install endpoint triggers installation
  and saves the resolved aria2c_path plus download_backend=aria2.
- Options UI adds an Install aria2 button, hides aria2-specific fields
  when another backend is selected, and updates status on completion.
- Includes .gitignore entry for tools/aria2/ so installed binaries are
  not committed.
2026-06-29 21:55:27 +02:00
Dariusz L 59e874c17f feat: improve CivitAI download support with session tokens and proper headers
- Add __init__.py: support civitai.red domain and session token cookies
- Add core/downloader.py: build_download_headers for consistent User-Agent,
  Accept, and CivitAI-specific Referer/Origin headers across download backends
- Resolve CivitAI API redirects before handing URLs to aria2
- Strip sensitive URL parameters from download logs
- Forward civitai_session_token from the frontend to the downloader
2026-06-28 03:42:41 +02:00
Dariusz L aa91d1410c feat: add aria2 download backend with pause/resume support
Integrates aria2c as an alternative download backend alongside the built-in
Python downloader. Adds settings for selecting the backend and configuring
the aria2c executable path, plus a status check button in the options panel.

Backend changes:
- core/downloader.py: aria2 daemon lifecycle, JSON-RPC client, pause/resume
  state machine, and transfer tracking
- core/settings.py: download_backend normalization and aria2c_path setting
- __init__.py: new API routes for pause, resume, and aria2 status checks

Frontend changes:
- options_methods.js: backend selector, aria2c path input, check button,
  and local storage persistence
- resolve_download_methods.js: pause/resume handlers, pending status
  transitions, and progress rendering for aria2 downloads
- queue_methods.js: queue panel pause/resume buttons, backend badge, and
  debounced panel updates
- search_panel.js: actionsHtml support for composite action buttons
2026-06-27 23:39:31 +02:00
Dariusz L b73cdb0f40 feat: improve category resolution and missing models browser
- Expand preferred categories to include clip_vision, ipadapter, sams, ultralytics
- Add clip_gguf alias for text_encoders in both backend and frontend
- Add explicit diffusion_models/text_encoders mappings in CATEGORY_MAP
- Refactor get_model_roots to skip custom_nodes/configs, use template key
  aliases, accept new folder keys from known categories, and normalize
  ultralytics paths (yolo/bbox, yolo/segm, yolo -> parent)
- Pass missing object through resolve_download_methods and search_panel
  so category selection can use widget hints
- Refactor frontend download target resolution:
  - getMissingSupportedDownloadCategories respects widget/type hints
  - getNodeTypeDownloadCategory splits exact-node map and adds
    normalizeNodeCategoryToken / getNodeTypeDownloadCategoryFromTokens
    for GGUF and future node families
  - getSourceResultDownloadCategory and _sourceSupportLevel accept
    missing for unified logic
- Prioritize dynamic widget category hints in workflow analyzer and
  include category_hints/folder_key_hints on all model refs
- Add folder count badges and path normalization in missing browser
  (normalizeMissingFolderRootPath handles category suffixes / bbox/segm)
2026-06-27 19:36:47 +02:00
Dariusz L e53d9f2a0b feat: generalize remote metadata lookup across CivitAI, CivArchive, and HuggingFace
Replace CivitAI-only search with a multi-source metadata resolution route using
SHA256 exact-match validation, dead-link filtering, and live download URL
probing. Unify sidecar metadata saving across all remote sources. Add
force-refresh support, local metadata import with preview detection, and
frontend source-aware UI controls.
2026-06-27 03:25:31 +02:00
Dariusz L 3cd302d4ee feat: implement local-only model info fetching and refactor model info dialog
- Add local_only endpoint mode: returns basic file info (filename, size, path,
  location, sha256) without querying CivitAI or computing file hashes
- Validate provided sha256/hash inputs (must be 64-char hex) and fall back to
  empty string when invalid
- Refactor monolithic info response builder into build_info_response() helper
  used across all result paths (local, CivitAI success, CivitAI fallback,
  not-found), reducing duplication
- Add get_model_info_for_file(local_only=True) early return in CivitAI source
  that builds a minimal local-only result without hash lookup
- Split frontend fetchModelInfoForDialog into:
  - fetchLocalModelInfoForDialog: opens dialog with local data immediately,
    marks civitai_checked=false
  - fetchCivitaiModelInfoForDialog: async CivitAI lookup triggered by user
- Add 'Fetch with CivitAI' button in info dialog when local file exists but
  CivitAI has not been checked yet
- Infer model_type from category in both backend and frontend to reduce
  reliance on upstream data shape
- Fix sidecar metadata condition (was skipping save when path already existed)
- Add cursor/disabled styles for info dialog CivitAI link button
2026-06-26 22:16:44 +02:00
Dariusz L 4e576cbd56 feat: add SHA256 hash calculation with cancellation, progress tracking, and remote size probing 2026-06-26 21:51:55 +02:00
Dariusz L c7d6b27fb0 feat: add search cancellation API and harden CivArchive source resolution
Add a /model_resolver/search-cancel/{progress_id} endpoint with a
SearchCancelled exception so in-flight source searches abort cleanly on
batch cancel. Track sourceProgressIds alongside sourceControllers so the
frontend can cancel backend progress polls. Rework CivArchive extraction
with robust __NEXT_DATA__ parsing, hash-page file resolution, model-link
scraping, confidence-based candidate filtering, and expanded type maps.
Add conftest.py for pytest path setup, update .gitignore for scratch/,
and refresh huggingface-author-index.json.
2026-06-26 00:55:38 +02:00
Dariusz L a320d4d805 refactor: consolidate duplicated logic into shared utilities
- Add get_comfy_root_path() to core/path_utils.py and use it in __init__.py and core/downloader.py instead of inline logic
- Add select_primary_model_file(), parse_size_to_bytes(), get_version_sort_key(), and check_credential_http() to core/type_utils.py
- Replace duplicated credential-check helpers in civitai.py, huggingface.py with check_credential_http()
- Replace duplicated size parsing and version sorting in civarchive.py, civitai.py with shared utilities
- Replace duplicated primary-file selection in civarchive.py, civitai.py, lora_manager_archive.py with select_primary_model_file()
- Replace inline tempfile-based JSON writes in popular.py with write_json_atomic()
- Remove dead reload_model_list() from model_list.py
- Fix workflow_analyzer.py hint prioritization for diffusion_models vs specific output hints
2026-06-25 18:54:59 +02:00
Dariusz L 6850ef1267 refactor: centralize type coercion and file I/O helpers to eliminate duplication 2026-06-25 17:33:59 +02:00
Dariusz L d979bbcf21 refactor: extract shared utilities and deduplicate metadata path definitions
- Add first_non_empty() and to_int() to core/type_utils.py
- Add get_filename_from_path() and METADATA_DIR to core/path_utils.py
- Replace inline implementations across resolver, downloader, and sources
with calls to the shared utilities
- Move strip_known_model_extension from resolver to matcher
2026-06-25 17:00:27 +02:00
Dariusz L 8655f92515 refactor: centralize category mapping and atomic JSON writing 2026-06-25 16:45:55 +02:00
Dariusz L 96036f0d6e refactor: DRY up source search functions and update HF author index
- Refactor __init__.py: extract run_source_search and execute_search_with_fallback
  helpers to eliminate duplicated progress-tracking, logging, and error-handling
  patterns across local, HuggingFace, CivitAI, CivArchive, and LoRA archive
  source search tasks; also fix URN variable shadowing in CivitAI/CivArchive

- Update metadata/huggingface-author-index.json with refreshed Comfy-Org
  author index (new repos and model entries)
2026-06-25 16:35:29 +02:00
Dariusz L 91d8e0cf3f Add return_success_on_error to API endpoints and import has_known_model_extension 2026-06-25 02:41:02 +02:00
Dariusz L 92e2601631 refactor: centralize API error handling and confidence calculation logic 2026-06-25 02:24:02 +02:00
Dariusz L 8f890f6992 refactor: centralize path utilities in core/path_utils
Extract repeated path normalization and identity logic from __init__.py,
core/resolver.py, core/downloader.py, and core/scanner.py into a shared
core/path_utils module. Also add a loading state to the local model picker
so it waits for models to load before rendering options.
2026-06-25 01:23:20 +02:00
Dariusz L 0b98054536 refactor: consolidate duplicated credential checks and local match fetch logic 2026-06-24 23:54:18 +02:00
Dariusz L 5445dd4ac0 feat: implement local hash matching via metadata sidecar files
Backend adds SHA256 normalization, metadata sidecar lookup, and hash fields across source modules. Frontend handles local_hash_matches in search responses and missing browser.
2026-06-24 21:59:10 +02:00
Dariusz L 8c0168ab76 feat: skip model analysis for empty or non-model workflows
Add early-return optimizations so both backend and frontend avoid unnecessary
URL extraction and disk scanning when a workflow has no nodes or contains
no potential model references.

Also tighten API input validation by rejecting non-object workflow_json.
2026-06-24 03:46:50 +02:00
Dariusz L 16a32de8de feat: improve model info dialog with clickable location, wider layout, and refined styling 2026-06-23 20:28:48 +02:00
Dariusz L 093b6d33a7 feat: enrich model info dialog with location, size, tags, and trained words from metadata 2026-06-23 19:47:49 +02:00
Dariusz L 7cd57c05cb improve folder path resolution with identity-based deduping and directory preference logic 2026-06-23 03:50:02 +02:00
Dariusz L 48bc3bf5f1 feat: wire progress callbacks into backend search sources 2026-06-23 03:33:28 +02:00
Dariusz L d8124d03da feat: add live backend search progress tracking
Frontend now polls a new backend endpoint (/model_resolver/search-progress/{progress_id})
to display real-time, per-source progress messages and percentages during model searches,
replacing static 'Searching...' indicators with granular status updates.
2026-06-23 02:23:45 +02:00
Dariusz L 1a4156f00d feat: add any-model fallback search for civitai, civarchive, and lora_manager_archive
When a model search returns no results for a specific base_model, retry the
same query with base_model_context=None so the provider can match any model.
Fallback matches are marked with any_model_match and base_model_fallback on
the result payload. The UI reports 'Found (Any Model)' when a result came
from this fallback path, and the cancel/progress button styling is tightened
to 16x16px with border-radius.
2026-06-20 00:53:51 +02:00
Dariusz L 936f1bed1a Add LoRA Manager-compatible sidecar metadata generation for downloaded models 2026-06-19 04:11:34 +02:00
Dariusz L 4f91accbab feat: improve model scan cache override, combo UX, and filename handling
- Add force_rescan end-to-end so callers can bypass the short-lived local model scan cache: backend entry in __init__.py and core/resolver.py, lifecycle_graph_methods forwards it through the workflow analysis body, and download_target_methods passes ?force=1 from the refresh flow.
- Rewrite combo dropdown lifecycle in missing_browser_methods with show/hide helpers, pointer-capture tracking, and a capture-phase outside-click listener so the list stays open during drag/hover and closes reliably otherwise.
- Animate refresh buttons (CSS keyframes + JS Web Animations) with reduced-motion fallback.
- Encode selected model path into the selected-bar data-model attribute and wire contextmenu so users can open the containing folder from the selection.
- Fix XSS risk in selected-bar label by escaping label HTML; change queue/selected Remove buttons to danger style.
- Increase max missing-browser model column width clamp from 430 to 820 and stop truncating missing-filename display; show full filename with tooltip instead.
- Update huggingface-author-index.json metadata timestamp/counts (1 file changed, 1 line).
2026-06-19 03:57:40 +02:00
Dariusz L 6de0668c4b feat: add sams and ultralytics model categories
Add support for Two new model categories with full frontend and backend
integration:

- sams: maps sam, sam_model, sam_models → sams category
- ultralytics: maps ultralytics, ultralytics_bbox, ultralytics_segm, yolo → ultralytics category

Covers CATEGORY_MAP, settings templates/aliases, workflow_analyzer node hints,
download target resolution (JS), and folder detection.
2026-06-19 00:44:36 +02:00
Dariusz L 32d437df50 Expand model category and node type recognition
Add support for additional ComfyUI model types including diffusers, gligen,
style models (latent/regular upscale), audio encoders, background removal,
frame interpolation, geometry estimation, detection, model patches/photomaker,
and optical flow models.

Backend changes:
- Extend CATEGORY_MAP, URN_TYPE_MAP, and node type hints in workflow analyzer
  to cover dozens of new loader node types and singular/plural aliases.
- Add MODEL_OUTPUT_TYPE_TO_CATEGORY and get_node_output_category_hint() so
  loader output types can contribute category hints.
- Make widget name matching normalize underscores, hyphens, and spaces.
- Improve directory discovery to enumerate folders from folder_paths while
  filtering unsupported categories.
- Include folders as model entries in directory scanning (with a diffusers
  guard requiring model_index.json).

Frontend changes:
- Update download target category maps, display names, token name maps, and
  per-node widget index lists to match the expanded backend coverage.
2026-06-19 00:35:06 +02:00
Dariusz L 47d286620f feat: add configurable download path system with templates and mappings
Implements a new download path configuration UI and backend logic:
- Add Subfolder fill mode: Suggested (existing), Template, Manual
- Add Base Model Path Mappings to override detected base-model folders
- Add Default Root settings to control root directory per category
- Add Detect From Existing Folders button that infers templates and mappings from local model folders
- Add backend /model_resolver/root-directories and /model_resolver/path-template-suggestions routes
- Refactor resolver settings into core/settings.py for shared frontend/backend use
- Extract path template inference logic into core/path_templates.py
- Update download panel, context menu, and loading flow to use new path settings
2026-06-19 00:09:58 +02:00
Dariusz L a4d5bd4c5c feat(logging): add frontend/backend log controls, unify console logging
- Add frontend_logs_enabled/frontend_log_level and backend_logs_enabled/backend_log_level
  toggles and selectors in the Options panel (web/resolver/views/options_methods.js)
- Sync frontend logging settings from localStorage / server settings into the
  frontend logger on startup and settings load (download_target_methods.js)
- Apply backend logging settings from server settings in the Python backend
  (__init__.py), loading control state on startup and after saves
- Make frontend Logger resilient to stored configs that lack an 'enabled' flag
  and to legacy/invalid moduleSettings levels (logger.js)
- Guard default module logger level initialization against undefined settings
  (log_funcs.js)
- Replace scattered console.log/usages with module-level log.debug calls across
  resolve_download_methods.js, model_resolver.js, and workflow_state_methods.js
- Bump log system package version to 1.5.6 (core + web)
2026-06-17 01:51:29 +02:00
Dariusz L 3dda91a39b Improve model path resolution with scanner-based exact-case matching
Pass available_models through to the workflow analyzer, enabling
case-sensitive relative path lookups against scanned model data.
Normalize paths for comparison while preserving letter casing.
Skip irrelevant categories (custom_nodes, configs) upfront.
Fall back gracefully when no scanner data is available.
Also ignore the tests/ directory in .gitignore.
2026-06-16 23:39:16 +02:00
Dariusz L f6c2d924a5 feat: show base directory only when subfolder values collide 2026-06-12 17:35:53 +02:00
Dariusz L 1cd9668db2 feat: track base_directory across download and subfolder flows; normalize clip to text_encoders 2026-06-12 17:23:59 +02:00
Dariusz L 91a998bea2 feat: generalize download subfolder suggestions for all model types
- Replace CivitAI-specific subfolder suggestion with broadened logic that derives candidates from model metadata, filename patterns, and existing folder structure
- Add getSuggestedModelSubfolderCandidates / getSuggestedExistingSubfolderByModelName helpers
- Normalize download category consistently via normalize_download_category in backend and frontend
- Backend now walks actual ComfyUI folder paths dynamically instead of hardcoded category mapping
- Rename frontend methods to applySuggestedDownloadSubfolder / forceSuggestedDownloadSubfolder
- Update Auto-fill subfolder tooltip to reflect expanded scope across all model categories
2026-06-12 17:02:28 +02:00
Dariusz L 7a1473b58e feat: normalize download category strings across backend and frontend
Add a shared normalize_download_category helper (with CATEGORY_MAP) in Python
and use it in get_download_directory to prefer redirected paths and to rank
diffusion_models above legacy unet folders. Mirror the same normalization in
the JS downloadTargetMethods, applying it consistently when coalescing
downloadDirectories, building dropdown keys, and reading/writing selections.
Also fix display labels to use 'diffusion_models' instead of 'unet'.
2026-06-12 16:42:58 +02:00
Dariusz L bbfd50a075 feat: expose download folder context through status indicators and notifications
Propagate the resolved download directory and file path from the backend
download flow into frontend progress UI: add \path\ and \directory\ fields
to download progress state, render right-clickable context menu entries on
progress bars and status messages, and allow opening the download folder
directly from completion or error notifications. Also update the HuggingFace
author index.
2026-06-12 16:32:48 +02:00
Dariusz L 2bdc3c2614 refactor: replace ad-hoc log formatting with structured helpers in search flow
Update search logging to use reusable formatters for values, sizes, and field
pairs, and add append-only extra kwargs so downstream logs stay on one line.
Tighten HuggingFace source messages, refresh the author index metadata, and
consolidate LoraManager/CivitAI log text.
2026-06-08 17:21:24 +02:00
Dariusz L 2a43597c9e feat: support model details for LoraManager archive downloads
Map lora_manager_archive source to Civitai on the details endpoint so LoraManager archive results can retrieve model metadata. Add detailsContext payload to resolve_download_methods and extend model_info_methods to accept lora_manager_archive as a valid source.
2026-06-08 04:49:41 +02:00
Dariusz L 65b4594d19 feat: add model details modal and source-aware context menus 2026-06-08 00:53:39 +02:00
Dariusz L 420b7e50c2 feat: plumb lora strength and active flags through workflow analysis for accurate model resolution 2026-06-07 17:52:28 +02:00
Dariusz L 9a99859a1c feat: plumb base_model_context through to civitai, civarchive, and lora manager search calls 2026-06-06 20:04:28 +02:00
Dariusz L ab59cd231f feat: add base model filtering across search sources, API routes, and frontend UI 2026-06-06 19:43:43 +02:00
Dariusz L 541881ce49 refactor: replace post-download re-analysis with targeted local match refresh
Replace the heavy autoResolveAfterDownload path (which re-analyzed the
entire workflow) with a more efficient refreshLocalMatchesForDownloadedMissing
call that queries /model_resolver/local-matches directly for the
specific downloaded file.

This removes the unnecessary re-analysis step, adds an explicit
force_rescan flag (coerced to bool on the backend for safe JSON
payload handling), propagates the missing model's category through
the activeDownloads tracking objects, and updates related UI text
to reflect the new narrower behavior.
2026-06-06 18:09:06 +02:00
Dariusz L 3d176b5e9a feat: propagate CivArchive search errors to UI status and tracking
Raise CivArchiveSearchError in civarchive backend, catch and store it in
source_results in the workflow analyzer, surface it in resolve_download_methods
as an 'error' status, and display error messages in search_panel progress
items and tooltips.
2026-06-06 17:13:18 +02:00
Dariusz L 57b8bc7cda feat: add force search (ignore cache) toggle to batch search menu
Adds a checkbox in the resolver footer search menu that bypasses all
search result caches for the duration of a batch search run.

Frontend (queue_methods.js, resolve_download_methods.js,
resolver_dialog.js, search_panel.js, resolver-main.css):
- Introduce checkbox-type footer menu items (type: 'checkbox').
- Add 'Force search (ignore cache)' checkbox to the search menu,
  wired to a new forceBatchSearch flag on ResolverManagerDialog.
- Pass forceSearch through searchMissingBatch and searchOnline to
  the backend API call.
- When the manual search button is clicked and results already exist
  for that model, forceSearch defaults to true.
- mergeSearchResults respects forceRefresh: if a forced search
  returns no results for a source, that source is cleared from the
  cached results.

Backend (__init__.py, core/sources/huggingface.py):
- force_search is read from the incoming API request and, when true,
  clears all active source search caches (local, huggingface,
  civitai, civarchive, lora_manager_archive) before searching.
- force_search is forwarded as force_refresh to
  search_huggingface_for_file, which folds it into the cache key so
  the cache is not hit, and propagates it to _get_author_index.
- Search result timestamps are no longer inherited from prior
  cached signatures when force_search is active; a fresh timestamp
  is written unconditionally.

Updated metadata/huggingface-author-index.json reflects the latest
cached author index after rebuild.
2026-06-06 17:02:25 +02:00
Dariusz L 57018b4e0f feat: add search result timestamps, enhanced cache clearing, and maintenance UI
Add searched_at timestamps to search results so cached entries are accurately tracked and preserved across merges. Expand the clear-search-cache command to also invalidate popular databases, model list, and the new result timestamp store. Frontend gains new cache-clearing methods and a full resolver cache reset for maintenance. The search panel and missing browser now show source result dates and toggle the search button between 'Search' and 'Search Again' when results already exist.
2026-06-06 16:38:30 +02:00