Commit Graph
97 Commits
Author SHA1 Message Date
Dariusz L f3cd3605a8 refactor: centralize type maps and normalize_download_category in type_utils
Move CATEGORY_MAP, normalize_download_category, URN_TYPE_MAP,
CIVITAI_API_TYPE_MAP, and CIVARCHIVE_API_TYPE_MAP into type_utils.py
so they are defined once and imported by settings, civitai, civarchive,
and workflow_analyzer instead of being duplicated across modules.
2026-06-25 18:03:28 +02:00
Dariusz L 6850ef1267 refactor: centralize type coercion and file I/O helpers to eliminate duplication 2026-06-25 17:33:59 +02:00
Dariusz L d979bbcf21 refactor: extract shared utilities and deduplicate metadata path definitions
- Add first_non_empty() and to_int() to core/type_utils.py
- Add get_filename_from_path() and METADATA_DIR to core/path_utils.py
- Replace inline implementations across resolver, downloader, and sources
with calls to the shared utilities
- Move strip_known_model_extension from resolver to matcher
2026-06-25 17:00:27 +02:00
Dariusz L 905faa9056 refactor: centralize MODEL_EXTENSIONS in type_utils to eliminate duplication 2026-06-25 16:53:32 +02:00
Dariusz L 8655f92515 refactor: centralize category mapping and atomic JSON writing 2026-06-25 16:45:55 +02:00
Dariusz L 91d8e0cf3f Add return_success_on_error to API endpoints and import has_known_model_extension 2026-06-25 02:41:02 +02:00
Dariusz L 92e2601631 refactor: centralize API error handling and confidence calculation logic 2026-06-25 02:24:02 +02:00
Dariusz L e4c3b0fb6f refactor: consolidate type and progress utilities
Add core/type_utils.py with as_dict/as_list helpers, and
core/progress.py with a unified report_progress() callback.

Replace inline duplicated implementations of _json_safe_metadata(),
_as_list(), normalize_filename(), and per-source _report_progress()
with calls to the shared modules across downloader, matcher, resolver,
and all source modules (CivitAI, CivArchive, HuggingFace, LoRA Manager).
Extend filename normalization to include dots as separators.
2026-06-25 02:09:47 +02:00
Dariusz L d1a5ccfedb refactor: centralize base model normalization and matching in matcher 2026-06-25 01:33:25 +02:00
Dariusz L 8f890f6992 refactor: centralize path utilities in core/path_utils
Extract repeated path normalization and identity logic from __init__.py,
core/resolver.py, core/downloader.py, and core/scanner.py into a shared
core/path_utils module. Also add a loading state to the local model picker
so it waits for models to load before rendering options.
2026-06-25 01:23:20 +02:00
Dariusz L 34638d517e refactor: share normalize_sha256 and parse_huggingface_url across modules 2026-06-25 00:03:34 +02:00
Dariusz L 5445dd4ac0 feat: implement local hash matching via metadata sidecar files
Backend adds SHA256 normalization, metadata sidecar lookup, and hash fields across source modules. Frontend handles local_hash_matches in search responses and missing browser.
2026-06-24 21:59:10 +02:00
Dariusz L 8867058312 feat: improve download folder selection UX with category tooltips and dynamic widget category hints
- core/workflow_analyzer.py: add dynamic widget category hint extraction via threaded cache and sentinel-prefix matching; get_model_widget_category_hints now returns all inferred categories including dynamic ones, improving model folder detection for workflows with encoded category metadata.
- web: add ? help icon with context-aware tooltip to download folder selector explaining why a category was auto-selected (preserved manual choice, node type hint, workflow-detected category, default fallback, or best available). Tooltip refreshes live as the user changes the folder selection.
- web/css/resolver-main.css: style the new category help badge.
- metadata/huggingface-author-index.json: update index entry.
2026-06-24 21:01:21 +02:00
Dariusz L c5c3490210 feat: add auto-refresh of ComfyUI model catalog after Apply 2026-06-24 19:22:55 +02:00
Dariusz L 8c0168ab76 feat: skip model analysis for empty or non-model workflows
Add early-return optimizations so both backend and frontend avoid unnecessary
URL extraction and disk scanning when a workflow has no nodes or contains
no potential model references.

Also tighten API input validation by rejecting non-object workflow_json.
2026-06-24 03:46:50 +02:00
Dariusz L b292411b93 feat: enhance model info image preview with CivitAI badge, metadata fields, and linked resources
- Add CivitAI badge overlay on example images with icon and direct link
- Extract denoise, scheduler, and cfgScale from metadata in civitai.py
- Support additionalResources alongside resources in both parser paths
- Redesign resource display with clickable links, version badges, and type/weight pills
- Drop prompt tags section and getInfoImagePromptTags; focus preview on generation data and resources
- Include full metadata rows and resources in copy-to-clipboard output
- Add matching CSS for badge, resource layout, and interactive states
2026-06-23 20:48:08 +02:00
Dariusz L 093b6d33a7 feat: enrich model info dialog with location, size, tags, and trained words from metadata 2026-06-23 19:47:49 +02:00
Dariusz L 48bc3bf5f1 feat: wire progress callbacks into backend search sources 2026-06-23 03:33:28 +02:00
Dariusz L b086ffd106 Improve symlink/junction handling and add RapidFuzz for faster fuzzy matching
- Add rapidfuzz as optional dependency with SequenceMatcher fallback
- Use stable path identity (realpath) for deduplication in scanner and resolver
- Prevent duplicate scans and duplicate model entries across symlinks/junctions
- Improve matching performance with RapidFuzz when available
2026-06-22 02:36:40 +02:00
Dariusz L a499af3345 Enable local model matching for resolved models and prevent symlink/junction scan loops 2026-06-22 02:21:24 +02:00
Dariusz L 590b4a4969 feat(missing-browser): add toggle to show resolved models
Report resolved (already-existing) models alongside missing ones in the
missing-browser view. Resolved models are hidden by default; a new
'Show resolved' toggle in the toolbar reveals them with a badge and
reduced-opacity row styling. The toggle state is persisted in
localStorage.

Backend: include resolved_models and total_resolved in the analysis
payload so the frontend can merge them into the display list.

Frontend: normalize resolved workflow models into the missing-models
display shape, compute visibility-aware counts, and refresh the
missing-browser view on queue mutating actions.
2026-06-22 01:34:55 +02:00
Dariusz L 92be0d6213 Improve base model alias resolution and update model metadata
- Add version-trimmed alias generation to handle trailing version zeros (e.g. '4.0' -> '4')
- Track newly added base model names in 'new_models_added_list' field
- Include Ideogram 4.0, Boogu, and PolyGen in base-models registry
- Update web resolver to match aliases using both original and version-trimmed tokens
- Enhance base models update UI to show which new models were added
2026-06-21 02:29:42 +02:00
Dariusz L 66bbaedf66 Verify SHA256 for existing downloads and surface already_exists in UI
core/downloader.py: When a file already exists at dest_path, extract the expected
SHA256 from metadata, prefers a trusted LoRA Manager sidecar hash, falls back to
calculating the file hash, and returns already_exists=true plus sha256_source
when the hashes match. If the hashes differ, the download is skipped with a
clear warning instead of silently re-downloading.

web/resolver/actions/resolve_download_methods.js: Thread already_exists through
resolveDownloadMethods so notifications, progress messages, and snapshot payloads
reflect 'Already downloaded' when the backend determined the model was a SHA256
match.
2026-06-19 04:27:53 +02:00
Dariusz L 936f1bed1a Add LoRA Manager-compatible sidecar metadata generation for downloaded models 2026-06-19 04:11:34 +02:00
Dariusz L 4f91accbab feat: improve model scan cache override, combo UX, and filename handling
- Add force_rescan end-to-end so callers can bypass the short-lived local model scan cache: backend entry in __init__.py and core/resolver.py, lifecycle_graph_methods forwards it through the workflow analysis body, and download_target_methods passes ?force=1 from the refresh flow.
- Rewrite combo dropdown lifecycle in missing_browser_methods with show/hide helpers, pointer-capture tracking, and a capture-phase outside-click listener so the list stays open during drag/hover and closes reliably otherwise.
- Animate refresh buttons (CSS keyframes + JS Web Animations) with reduced-motion fallback.
- Encode selected model path into the selected-bar data-model attribute and wire contextmenu so users can open the containing folder from the selection.
- Fix XSS risk in selected-bar label by escaping label HTML; change queue/selected Remove buttons to danger style.
- Increase max missing-browser model column width clamp from 430 to 820 and stop truncating missing-filename display; show full filename with tooltip instead.
- Update huggingface-author-index.json metadata timestamp/counts (1 file changed, 1 line).
2026-06-19 03:57:40 +02:00
Dariusz L 5f1e814751 feat: support subgraph-aware node location and widget model context
- workflow_analyzer: refactor widget category hint resolution and add widget name candidate extraction across widgets, inputs, proxyWidgets, and localized names; expose new helpers get_widget_name_candidates/get_widget_name_hint
- workflow_analyzer: add get_model_widget_category_hint and get_effective_model_category_hint to unify category fallback logic
- workflow_analyzer: include widget_name on model refs emitted by get_node_model_info
- workflow_analyzer: introduce promoted-widget context resolution (_build_promoted_widget_contexts, _find_promoted_widget_targets, etc.) and wire it into analyze_workflow_models so instance widgets and promoted widget locators are tracked with subgraph scope
- lifecycle_graph_methods: refactor locateNodeInGraph into subgraph-aware resolver using findLocateTarget/getGraphNodeById/findSubgraphById/findNodeInGraphTree; support targeting nodes inside specific subgraphs with proper canvas focus and notifications
- missing_browser_methods: use unified getMissingLocateTarget for locate button/display fields; pass subgraphId and isTopLevel to locateNodeInGraph; add promoted-widget-aware tooltips and canLocate handling
2026-06-19 01:39:15 +02:00
Dariusz L 6de0668c4b feat: add sams and ultralytics model categories
Add support for Two new model categories with full frontend and backend
integration:

- sams: maps sam, sam_model, sam_models → sams category
- ultralytics: maps ultralytics, ultralytics_bbox, ultralytics_segm, yolo → ultralytics category

Covers CATEGORY_MAP, settings templates/aliases, workflow_analyzer node hints,
download target resolution (JS), and folder detection.
2026-06-19 00:44:36 +02:00
Dariusz L 32d437df50 Expand model category and node type recognition
Add support for additional ComfyUI model types including diffusers, gligen,
style models (latent/regular upscale), audio encoders, background removal,
frame interpolation, geometry estimation, detection, model patches/photomaker,
and optical flow models.

Backend changes:
- Extend CATEGORY_MAP, URN_TYPE_MAP, and node type hints in workflow analyzer
  to cover dozens of new loader node types and singular/plural aliases.
- Add MODEL_OUTPUT_TYPE_TO_CATEGORY and get_node_output_category_hint() so
  loader output types can contribute category hints.
- Make widget name matching normalize underscores, hyphens, and spaces.
- Improve directory discovery to enumerate folders from folder_paths while
  filtering unsupported categories.
- Include folders as model entries in directory scanning (with a diffusers
  guard requiring model_index.json).

Frontend changes:
- Update download target category maps, display names, token name maps, and
  per-node widget index lists to match the expanded backend coverage.
2026-06-19 00:35:06 +02:00
Dariusz L 47d286620f feat: add configurable download path system with templates and mappings
Implements a new download path configuration UI and backend logic:
- Add Subfolder fill mode: Suggested (existing), Template, Manual
- Add Base Model Path Mappings to override detected base-model folders
- Add Default Root settings to control root directory per category
- Add Detect From Existing Folders button that infers templates and mappings from local model folders
- Add backend /model_resolver/root-directories and /model_resolver/path-template-suggestions routes
- Refactor resolver settings into core/settings.py for shared frontend/backend use
- Extract path template inference logic into core/path_templates.py
- Update download panel, context menu, and loading flow to use new path settings
2026-06-19 00:09:58 +02:00
Dariusz L c9dd0e47ac feat: improve model resolution for extensionless workflow values and add new node type support
- Add model_title match type for CivArchive and CivitAI sources, resolving
  extensionless workflow values by title similarity (threshold 82.0)
- Recognize new node types and widget indices across model categories
  (GGUF, CLIP, ControlNet, StyleModel, Hypernetwork, etc.)
- Scan known model combo widget indices even when the stored value has no
  file extension, treating non-placeholder strings as model references
- Strip known model extensions during workflow URL lookup so filename stems
  match regardless of extension
- Update HuggingFace author index metadata
2026-06-18 02:19:14 +02:00
Dariusz L 2f14026abc Fix: update badges, add GGUF model category mapping, rename panel title
- README.md: updated badge labelColor from 111827 to 1a1a1a for consistency
- core/workflow_analyzer.py: add LoaderGGUF and gguf_name mapping to diffusion_models
- web/resolver/model_resolver.js: rename panel title from 'Model Resolver' to 'Resolver'
2026-06-18 01:09:54 +02:00
Dariusz L a4d5bd4c5c feat(logging): add frontend/backend log controls, unify console logging
- Add frontend_logs_enabled/frontend_log_level and backend_logs_enabled/backend_log_level
  toggles and selectors in the Options panel (web/resolver/views/options_methods.js)
- Sync frontend logging settings from localStorage / server settings into the
  frontend logger on startup and settings load (download_target_methods.js)
- Apply backend logging settings from server settings in the Python backend
  (__init__.py), loading control state on startup and after saves
- Make frontend Logger resilient to stored configs that lack an 'enabled' flag
  and to legacy/invalid moduleSettings levels (logger.js)
- Guard default module logger level initialization against undefined settings
  (log_funcs.js)
- Replace scattered console.log/usages with module-level log.debug calls across
  resolve_download_methods.js, model_resolver.js, and workflow_state_methods.js
- Bump log system package version to 1.5.6 (core + web)
2026-06-17 01:51:29 +02:00
Dariusz L 3dda91a39b Improve model path resolution with scanner-based exact-case matching
Pass available_models through to the workflow analyzer, enabling
case-sensitive relative path lookups against scanned model data.
Normalize paths for comparison while preserving letter casing.
Skip irrelevant categories (custom_nodes, configs) upfront.
Fall back gracefully when no scanner data is available.
Also ignore the tests/ directory in .gitignore.
2026-06-16 23:39:16 +02:00
Dariusz L 1cd9668db2 feat: track base_directory across download and subfolder flows; normalize clip to text_encoders 2026-06-12 17:23:59 +02:00
Dariusz L 7a1473b58e feat: normalize download category strings across backend and frontend
Add a shared normalize_download_category helper (with CATEGORY_MAP) in Python
and use it in get_download_directory to prefer redirected paths and to rank
diffusion_models above legacy unet folders. Mirror the same normalization in
the JS downloadTargetMethods, applying it consistently when coalescing
downloadDirectories, building dropdown keys, and reading/writing selections.
Also fix display labels to use 'diffusion_models' instead of 'unet'.
2026-06-12 16:42:58 +02:00
Dariusz L bbfd50a075 feat: expose download folder context through status indicators and notifications
Propagate the resolved download directory and file path from the backend
download flow into frontend progress UI: add \path\ and \directory\ fields
to download progress state, render right-clickable context menu entries on
progress bars and status messages, and allow opening the download folder
directly from completion or error notifications. Also update the HuggingFace
author index.
2026-06-12 16:32:48 +02:00
Dariusz L 2bdc3c2614 refactor: replace ad-hoc log formatting with structured helpers in search flow
Update search logging to use reusable formatters for values, sizes, and field
pairs, and add append-only extra kwargs so downstream logs stay on one line.
Tighten HuggingFace source messages, refresh the author index metadata, and
consolidate LoraManager/CivitAI log text.
2026-06-08 17:21:24 +02:00
Dariusz L ae5060317a refactor: replace print calls with structured logging in downloader 2026-06-08 04:52:33 +02:00
Dariusz L 93161add4d feat: add comprehensive file size extraction for Civitai/CivArchive payloads 2026-06-08 03:32:37 +02:00
Dariusz L 6e2f1b3101 feat: enhance CivArchive resilience and mirror display
backend:
- add 429 retry handling with backoff in CivArchive API requests
- add page-scraping fallback for CivArchive model payloads
- normalize CivArchive mirrors with dead/gated/paid/sha256 metadata
- include file metadata and mirror data in normalized archive files
- propagate additional dead/missing status field aliases
- include metadata in normalized CivitAI files

frontend:
- render mirror lists with source grouping, dead-link indicators, and skull icons
- add file badges (primary, type, format, size, fp, mirror count, id)
- improve secondary info in resolve/download methods to include base model
- enhance search panel secondary info with base model and filename
- on selecting a version with empty files, fetch fresh version details
- propagate base model context to CivitAI search state and display labels
2026-06-08 03:03:53 +02:00
Dariusz L 65b4594d19 feat: add model details modal and source-aware context menus 2026-06-08 00:53:39 +02:00
Dariusz L 87fabdd9d9 feat: add image preview modal and extended CivitAI image metadata
Adds a fullscreen image preview dialog to the model info panel with
navigation, copy-prompt support, generation metadata display, and
referenced resources listing.

Extends CivitAI backend extraction to handle varied metadata key names
(negativePrompt, Clip skip, Model, etc.) and surfaces clip_skip, width,
height, tags, resources, and raw metadata through the new preview UI.

Renames z-index variable to reserve a dedicated layer for the preview
backdrop.
2026-06-07 18:57:03 +02:00
Dariusz L 420b7e50c2 feat: plumb lora strength and active flags through workflow analysis for accurate model resolution 2026-06-07 17:52:28 +02:00
Dariusz L 7224d87135 fix: replace substring alias matching with exact membership check in _base_model_matches
Uses an exact alias membership check instead of substring matching to prevent
false positives, e.g. 'zimage' alias no longer incorrectly matches
'zimagebase'.
2026-06-07 14:40:44 +02:00
Dariusz L ab59cd231f feat: add base model filtering across search sources, API routes, and frontend UI 2026-06-06 19:43:43 +02:00
Dariusz L 541881ce49 refactor: replace post-download re-analysis with targeted local match refresh
Replace the heavy autoResolveAfterDownload path (which re-analyzed the
entire workflow) with a more efficient refreshLocalMatchesForDownloadedMissing
call that queries /model_resolver/local-matches directly for the
specific downloaded file.

This removes the unnecessary re-analysis step, adds an explicit
force_rescan flag (coerced to bool on the backend for safe JSON
payload handling), propagates the missing model's category through
the activeDownloads tracking objects, and updates related UI text
to reflect the new narrower behavior.
2026-06-06 18:09:06 +02:00
Dariusz L 3d176b5e9a feat: propagate CivArchive search errors to UI status and tracking
Raise CivArchiveSearchError in civarchive backend, catch and store it in
source_results in the workflow analyzer, surface it in resolve_download_methods
as an 'error' status, and display error messages in search_panel progress
items and tooltips.
2026-06-06 17:13:18 +02:00
Dariusz L 57b8bc7cda feat: add force search (ignore cache) toggle to batch search menu
Adds a checkbox in the resolver footer search menu that bypasses all
search result caches for the duration of a batch search run.

Frontend (queue_methods.js, resolve_download_methods.js,
resolver_dialog.js, search_panel.js, resolver-main.css):
- Introduce checkbox-type footer menu items (type: 'checkbox').
- Add 'Force search (ignore cache)' checkbox to the search menu,
  wired to a new forceBatchSearch flag on ResolverManagerDialog.
- Pass forceSearch through searchMissingBatch and searchOnline to
  the backend API call.
- When the manual search button is clicked and results already exist
  for that model, forceSearch defaults to true.
- mergeSearchResults respects forceRefresh: if a forced search
  returns no results for a source, that source is cleared from the
  cached results.

Backend (__init__.py, core/sources/huggingface.py):
- force_search is read from the incoming API request and, when true,
  clears all active source search caches (local, huggingface,
  civitai, civarchive, lora_manager_archive) before searching.
- force_search is forwarded as force_refresh to
  search_huggingface_for_file, which folds it into the cache key so
  the cache is not hit, and propagates it to _get_author_index.
- Search result timestamps are no longer inherited from prior
  cached signatures when force_search is active; a fresh timestamp
  is written unconditionally.

Updated metadata/huggingface-author-index.json reflects the latest
cached author index after rebuild.
2026-06-06 17:02:25 +02:00
Dariusz L 57018b4e0f feat: add search result timestamps, enhanced cache clearing, and maintenance UI
Add searched_at timestamps to search results so cached entries are accurately tracked and preserved across merges. Expand the clear-search-cache command to also invalidate popular databases, model list, and the new result timestamp store. Frontend gains new cache-clearing methods and a full resolver cache reset for maintenance. The search panel and missing browser now show source result dates and toggle the search button between 'Search' and 'Search Again' when results already exist.
2026-06-06 16:38:30 +02:00
Dariusz L 827017f7a0 Revert "revert: remove server-side settings persistence and revert to localStorage-only"
This reverts commit 888ab171c3.
2026-06-05 06:03:41 +02:00