Commit Graph
104 Commits
Author SHA1 Message Date
Dariusz L e53d9f2a0b feat: generalize remote metadata lookup across CivitAI, CivArchive, and HuggingFace
Replace CivitAI-only search with a multi-source metadata resolution route using
SHA256 exact-match validation, dead-link filtering, and live download URL
probing. Unify sidecar metadata saving across all remote sources. Add
force-refresh support, local metadata import with preview detection, and
frontend source-aware UI controls.
2026-06-27 03:25:31 +02:00
Dariusz L 3cd302d4ee feat: implement local-only model info fetching and refactor model info dialog
- Add local_only endpoint mode: returns basic file info (filename, size, path,
  location, sha256) without querying CivitAI or computing file hashes
- Validate provided sha256/hash inputs (must be 64-char hex) and fall back to
  empty string when invalid
- Refactor monolithic info response builder into build_info_response() helper
  used across all result paths (local, CivitAI success, CivitAI fallback,
  not-found), reducing duplication
- Add get_model_info_for_file(local_only=True) early return in CivitAI source
  that builds a minimal local-only result without hash lookup
- Split frontend fetchModelInfoForDialog into:
  - fetchLocalModelInfoForDialog: opens dialog with local data immediately,
    marks civitai_checked=false
  - fetchCivitaiModelInfoForDialog: async CivitAI lookup triggered by user
- Add 'Fetch with CivitAI' button in info dialog when local file exists but
  CivitAI has not been checked yet
- Infer model_type from category in both backend and frontend to reduce
  reliance on upstream data shape
- Fix sidecar metadata condition (was skipping save when path already existed)
- Add cursor/disabled styles for info dialog CivitAI link button
2026-06-26 22:16:44 +02:00
Dariusz L 4e576cbd56 feat: add SHA256 hash calculation with cancellation, progress tracking, and remote size probing 2026-06-26 21:51:55 +02:00
Dariusz L 57716ce9f9 feat(civarchive): add remote file size probing, hash page model card extraction, and model-type filtering 2026-06-26 20:50:37 +02:00
Dariusz L e386808dd4 feat: add CivitAI link, file matching, and mirror selection to model details
Add a direct "View on CivitAI" action button when a model has CivitAI IDs,
auto-detect the matched file via hash/URL/filename, surface the preferred
mirror, and render file entries as collapsible details with summary badges.
Normalize CSS formatting (rgba spacing, rule separators) for consistency.
2026-06-26 01:44:15 +02:00
Dariusz L c7d6b27fb0 feat: add search cancellation API and harden CivArchive source resolution
Add a /model_resolver/search-cancel/{progress_id} endpoint with a
SearchCancelled exception so in-flight source searches abort cleanly on
batch cancel. Track sourceProgressIds alongside sourceControllers so the
frontend can cancel backend progress polls. Rework CivArchive extraction
with robust __NEXT_DATA__ parsing, hash-page file resolution, model-link
scraping, confidence-based candidate filtering, and expanded type maps.
Add conftest.py for pytest path setup, update .gitignore for scratch/,
and refresh huggingface-author-index.json.
2026-06-26 00:55:38 +02:00
Dariusz L a320d4d805 refactor: consolidate duplicated logic into shared utilities
- Add get_comfy_root_path() to core/path_utils.py and use it in __init__.py and core/downloader.py instead of inline logic
- Add select_primary_model_file(), parse_size_to_bytes(), get_version_sort_key(), and check_credential_http() to core/type_utils.py
- Replace duplicated credential-check helpers in civitai.py, huggingface.py with check_credential_http()
- Replace duplicated size parsing and version sorting in civarchive.py, civitai.py with shared utilities
- Replace duplicated primary-file selection in civarchive.py, civitai.py, lora_manager_archive.py with select_primary_model_file()
- Replace inline tempfile-based JSON writes in popular.py with write_json_atomic()
- Remove dead reload_model_list() from model_list.py
- Fix workflow_analyzer.py hint prioritization for diffusion_models vs specific output hints
2026-06-25 18:54:59 +02:00
Dariusz L f3cd3605a8 refactor: centralize type maps and normalize_download_category in type_utils
Move CATEGORY_MAP, normalize_download_category, URN_TYPE_MAP,
CIVITAI_API_TYPE_MAP, and CIVARCHIVE_API_TYPE_MAP into type_utils.py
so they are defined once and imported by settings, civitai, civarchive,
and workflow_analyzer instead of being duplicated across modules.
2026-06-25 18:03:28 +02:00
Dariusz L 6850ef1267 refactor: centralize type coercion and file I/O helpers to eliminate duplication 2026-06-25 17:33:59 +02:00
Dariusz L d979bbcf21 refactor: extract shared utilities and deduplicate metadata path definitions
- Add first_non_empty() and to_int() to core/type_utils.py
- Add get_filename_from_path() and METADATA_DIR to core/path_utils.py
- Replace inline implementations across resolver, downloader, and sources
with calls to the shared utilities
- Move strip_known_model_extension from resolver to matcher
2026-06-25 17:00:27 +02:00
Dariusz L 905faa9056 refactor: centralize MODEL_EXTENSIONS in type_utils to eliminate duplication 2026-06-25 16:53:32 +02:00
Dariusz L 8655f92515 refactor: centralize category mapping and atomic JSON writing 2026-06-25 16:45:55 +02:00
Dariusz L 91d8e0cf3f Add return_success_on_error to API endpoints and import has_known_model_extension 2026-06-25 02:41:02 +02:00
Dariusz L 92e2601631 refactor: centralize API error handling and confidence calculation logic 2026-06-25 02:24:02 +02:00
Dariusz L e4c3b0fb6f refactor: consolidate type and progress utilities
Add core/type_utils.py with as_dict/as_list helpers, and
core/progress.py with a unified report_progress() callback.

Replace inline duplicated implementations of _json_safe_metadata(),
_as_list(), normalize_filename(), and per-source _report_progress()
with calls to the shared modules across downloader, matcher, resolver,
and all source modules (CivitAI, CivArchive, HuggingFace, LoRA Manager).
Extend filename normalization to include dots as separators.
2026-06-25 02:09:47 +02:00
Dariusz L d1a5ccfedb refactor: centralize base model normalization and matching in matcher 2026-06-25 01:33:25 +02:00
Dariusz L 8f890f6992 refactor: centralize path utilities in core/path_utils
Extract repeated path normalization and identity logic from __init__.py,
core/resolver.py, core/downloader.py, and core/scanner.py into a shared
core/path_utils module. Also add a loading state to the local model picker
so it waits for models to load before rendering options.
2026-06-25 01:23:20 +02:00
Dariusz L 34638d517e refactor: share normalize_sha256 and parse_huggingface_url across modules 2026-06-25 00:03:34 +02:00
Dariusz L 5445dd4ac0 feat: implement local hash matching via metadata sidecar files
Backend adds SHA256 normalization, metadata sidecar lookup, and hash fields across source modules. Frontend handles local_hash_matches in search responses and missing browser.
2026-06-24 21:59:10 +02:00
Dariusz L 8867058312 feat: improve download folder selection UX with category tooltips and dynamic widget category hints
- core/workflow_analyzer.py: add dynamic widget category hint extraction via threaded cache and sentinel-prefix matching; get_model_widget_category_hints now returns all inferred categories including dynamic ones, improving model folder detection for workflows with encoded category metadata.
- web: add ? help icon with context-aware tooltip to download folder selector explaining why a category was auto-selected (preserved manual choice, node type hint, workflow-detected category, default fallback, or best available). Tooltip refreshes live as the user changes the folder selection.
- web/css/resolver-main.css: style the new category help badge.
- metadata/huggingface-author-index.json: update index entry.
2026-06-24 21:01:21 +02:00
Dariusz L c5c3490210 feat: add auto-refresh of ComfyUI model catalog after Apply 2026-06-24 19:22:55 +02:00
Dariusz L 8c0168ab76 feat: skip model analysis for empty or non-model workflows
Add early-return optimizations so both backend and frontend avoid unnecessary
URL extraction and disk scanning when a workflow has no nodes or contains
no potential model references.

Also tighten API input validation by rejecting non-object workflow_json.
2026-06-24 03:46:50 +02:00
Dariusz L b292411b93 feat: enhance model info image preview with CivitAI badge, metadata fields, and linked resources
- Add CivitAI badge overlay on example images with icon and direct link
- Extract denoise, scheduler, and cfgScale from metadata in civitai.py
- Support additionalResources alongside resources in both parser paths
- Redesign resource display with clickable links, version badges, and type/weight pills
- Drop prompt tags section and getInfoImagePromptTags; focus preview on generation data and resources
- Include full metadata rows and resources in copy-to-clipboard output
- Add matching CSS for badge, resource layout, and interactive states
2026-06-23 20:48:08 +02:00
Dariusz L 093b6d33a7 feat: enrich model info dialog with location, size, tags, and trained words from metadata 2026-06-23 19:47:49 +02:00
Dariusz L 48bc3bf5f1 feat: wire progress callbacks into backend search sources 2026-06-23 03:33:28 +02:00
Dariusz L b086ffd106 Improve symlink/junction handling and add RapidFuzz for faster fuzzy matching
- Add rapidfuzz as optional dependency with SequenceMatcher fallback
- Use stable path identity (realpath) for deduplication in scanner and resolver
- Prevent duplicate scans and duplicate model entries across symlinks/junctions
- Improve matching performance with RapidFuzz when available
2026-06-22 02:36:40 +02:00
Dariusz L a499af3345 Enable local model matching for resolved models and prevent symlink/junction scan loops 2026-06-22 02:21:24 +02:00
Dariusz L 590b4a4969 feat(missing-browser): add toggle to show resolved models
Report resolved (already-existing) models alongside missing ones in the
missing-browser view. Resolved models are hidden by default; a new
'Show resolved' toggle in the toolbar reveals them with a badge and
reduced-opacity row styling. The toggle state is persisted in
localStorage.

Backend: include resolved_models and total_resolved in the analysis
payload so the frontend can merge them into the display list.

Frontend: normalize resolved workflow models into the missing-models
display shape, compute visibility-aware counts, and refresh the
missing-browser view on queue mutating actions.
2026-06-22 01:34:55 +02:00
Dariusz L 92be0d6213 Improve base model alias resolution and update model metadata
- Add version-trimmed alias generation to handle trailing version zeros (e.g. '4.0' -> '4')
- Track newly added base model names in 'new_models_added_list' field
- Include Ideogram 4.0, Boogu, and PolyGen in base-models registry
- Update web resolver to match aliases using both original and version-trimmed tokens
- Enhance base models update UI to show which new models were added
2026-06-21 02:29:42 +02:00
Dariusz L 66bbaedf66 Verify SHA256 for existing downloads and surface already_exists in UI
core/downloader.py: When a file already exists at dest_path, extract the expected
SHA256 from metadata, prefers a trusted LoRA Manager sidecar hash, falls back to
calculating the file hash, and returns already_exists=true plus sha256_source
when the hashes match. If the hashes differ, the download is skipped with a
clear warning instead of silently re-downloading.

web/resolver/actions/resolve_download_methods.js: Thread already_exists through
resolveDownloadMethods so notifications, progress messages, and snapshot payloads
reflect 'Already downloaded' when the backend determined the model was a SHA256
match.
2026-06-19 04:27:53 +02:00
Dariusz L 936f1bed1a Add LoRA Manager-compatible sidecar metadata generation for downloaded models 2026-06-19 04:11:34 +02:00
Dariusz L 4f91accbab feat: improve model scan cache override, combo UX, and filename handling
- Add force_rescan end-to-end so callers can bypass the short-lived local model scan cache: backend entry in __init__.py and core/resolver.py, lifecycle_graph_methods forwards it through the workflow analysis body, and download_target_methods passes ?force=1 from the refresh flow.
- Rewrite combo dropdown lifecycle in missing_browser_methods with show/hide helpers, pointer-capture tracking, and a capture-phase outside-click listener so the list stays open during drag/hover and closes reliably otherwise.
- Animate refresh buttons (CSS keyframes + JS Web Animations) with reduced-motion fallback.
- Encode selected model path into the selected-bar data-model attribute and wire contextmenu so users can open the containing folder from the selection.
- Fix XSS risk in selected-bar label by escaping label HTML; change queue/selected Remove buttons to danger style.
- Increase max missing-browser model column width clamp from 430 to 820 and stop truncating missing-filename display; show full filename with tooltip instead.
- Update huggingface-author-index.json metadata timestamp/counts (1 file changed, 1 line).
2026-06-19 03:57:40 +02:00
Dariusz L 5f1e814751 feat: support subgraph-aware node location and widget model context
- workflow_analyzer: refactor widget category hint resolution and add widget name candidate extraction across widgets, inputs, proxyWidgets, and localized names; expose new helpers get_widget_name_candidates/get_widget_name_hint
- workflow_analyzer: add get_model_widget_category_hint and get_effective_model_category_hint to unify category fallback logic
- workflow_analyzer: include widget_name on model refs emitted by get_node_model_info
- workflow_analyzer: introduce promoted-widget context resolution (_build_promoted_widget_contexts, _find_promoted_widget_targets, etc.) and wire it into analyze_workflow_models so instance widgets and promoted widget locators are tracked with subgraph scope
- lifecycle_graph_methods: refactor locateNodeInGraph into subgraph-aware resolver using findLocateTarget/getGraphNodeById/findSubgraphById/findNodeInGraphTree; support targeting nodes inside specific subgraphs with proper canvas focus and notifications
- missing_browser_methods: use unified getMissingLocateTarget for locate button/display fields; pass subgraphId and isTopLevel to locateNodeInGraph; add promoted-widget-aware tooltips and canLocate handling
2026-06-19 01:39:15 +02:00
Dariusz L 6de0668c4b feat: add sams and ultralytics model categories
Add support for Two new model categories with full frontend and backend
integration:

- sams: maps sam, sam_model, sam_models → sams category
- ultralytics: maps ultralytics, ultralytics_bbox, ultralytics_segm, yolo → ultralytics category

Covers CATEGORY_MAP, settings templates/aliases, workflow_analyzer node hints,
download target resolution (JS), and folder detection.
2026-06-19 00:44:36 +02:00
Dariusz L 32d437df50 Expand model category and node type recognition
Add support for additional ComfyUI model types including diffusers, gligen,
style models (latent/regular upscale), audio encoders, background removal,
frame interpolation, geometry estimation, detection, model patches/photomaker,
and optical flow models.

Backend changes:
- Extend CATEGORY_MAP, URN_TYPE_MAP, and node type hints in workflow analyzer
  to cover dozens of new loader node types and singular/plural aliases.
- Add MODEL_OUTPUT_TYPE_TO_CATEGORY and get_node_output_category_hint() so
  loader output types can contribute category hints.
- Make widget name matching normalize underscores, hyphens, and spaces.
- Improve directory discovery to enumerate folders from folder_paths while
  filtering unsupported categories.
- Include folders as model entries in directory scanning (with a diffusers
  guard requiring model_index.json).

Frontend changes:
- Update download target category maps, display names, token name maps, and
  per-node widget index lists to match the expanded backend coverage.
2026-06-19 00:35:06 +02:00
Dariusz L 47d286620f feat: add configurable download path system with templates and mappings
Implements a new download path configuration UI and backend logic:
- Add Subfolder fill mode: Suggested (existing), Template, Manual
- Add Base Model Path Mappings to override detected base-model folders
- Add Default Root settings to control root directory per category
- Add Detect From Existing Folders button that infers templates and mappings from local model folders
- Add backend /model_resolver/root-directories and /model_resolver/path-template-suggestions routes
- Refactor resolver settings into core/settings.py for shared frontend/backend use
- Extract path template inference logic into core/path_templates.py
- Update download panel, context menu, and loading flow to use new path settings
2026-06-19 00:09:58 +02:00
Dariusz L c9dd0e47ac feat: improve model resolution for extensionless workflow values and add new node type support
- Add model_title match type for CivArchive and CivitAI sources, resolving
  extensionless workflow values by title similarity (threshold 82.0)
- Recognize new node types and widget indices across model categories
  (GGUF, CLIP, ControlNet, StyleModel, Hypernetwork, etc.)
- Scan known model combo widget indices even when the stored value has no
  file extension, treating non-placeholder strings as model references
- Strip known model extensions during workflow URL lookup so filename stems
  match regardless of extension
- Update HuggingFace author index metadata
2026-06-18 02:19:14 +02:00
Dariusz L 2f14026abc Fix: update badges, add GGUF model category mapping, rename panel title
- README.md: updated badge labelColor from 111827 to 1a1a1a for consistency
- core/workflow_analyzer.py: add LoaderGGUF and gguf_name mapping to diffusion_models
- web/resolver/model_resolver.js: rename panel title from 'Model Resolver' to 'Resolver'
2026-06-18 01:09:54 +02:00
Dariusz L a4d5bd4c5c feat(logging): add frontend/backend log controls, unify console logging
- Add frontend_logs_enabled/frontend_log_level and backend_logs_enabled/backend_log_level
  toggles and selectors in the Options panel (web/resolver/views/options_methods.js)
- Sync frontend logging settings from localStorage / server settings into the
  frontend logger on startup and settings load (download_target_methods.js)
- Apply backend logging settings from server settings in the Python backend
  (__init__.py), loading control state on startup and after saves
- Make frontend Logger resilient to stored configs that lack an 'enabled' flag
  and to legacy/invalid moduleSettings levels (logger.js)
- Guard default module logger level initialization against undefined settings
  (log_funcs.js)
- Replace scattered console.log/usages with module-level log.debug calls across
  resolve_download_methods.js, model_resolver.js, and workflow_state_methods.js
- Bump log system package version to 1.5.6 (core + web)
2026-06-17 01:51:29 +02:00
Dariusz L 3dda91a39b Improve model path resolution with scanner-based exact-case matching
Pass available_models through to the workflow analyzer, enabling
case-sensitive relative path lookups against scanned model data.
Normalize paths for comparison while preserving letter casing.
Skip irrelevant categories (custom_nodes, configs) upfront.
Fall back gracefully when no scanner data is available.
Also ignore the tests/ directory in .gitignore.
2026-06-16 23:39:16 +02:00
Dariusz L 1cd9668db2 feat: track base_directory across download and subfolder flows; normalize clip to text_encoders 2026-06-12 17:23:59 +02:00
Dariusz L 7a1473b58e feat: normalize download category strings across backend and frontend
Add a shared normalize_download_category helper (with CATEGORY_MAP) in Python
and use it in get_download_directory to prefer redirected paths and to rank
diffusion_models above legacy unet folders. Mirror the same normalization in
the JS downloadTargetMethods, applying it consistently when coalescing
downloadDirectories, building dropdown keys, and reading/writing selections.
Also fix display labels to use 'diffusion_models' instead of 'unet'.
2026-06-12 16:42:58 +02:00
Dariusz L bbfd50a075 feat: expose download folder context through status indicators and notifications
Propagate the resolved download directory and file path from the backend
download flow into frontend progress UI: add \path\ and \directory\ fields
to download progress state, render right-clickable context menu entries on
progress bars and status messages, and allow opening the download folder
directly from completion or error notifications. Also update the HuggingFace
author index.
2026-06-12 16:32:48 +02:00
Dariusz L 2bdc3c2614 refactor: replace ad-hoc log formatting with structured helpers in search flow
Update search logging to use reusable formatters for values, sizes, and field
pairs, and add append-only extra kwargs so downstream logs stay on one line.
Tighten HuggingFace source messages, refresh the author index metadata, and
consolidate LoraManager/CivitAI log text.
2026-06-08 17:21:24 +02:00
Dariusz L ae5060317a refactor: replace print calls with structured logging in downloader 2026-06-08 04:52:33 +02:00
Dariusz L 93161add4d feat: add comprehensive file size extraction for Civitai/CivArchive payloads 2026-06-08 03:32:37 +02:00
Dariusz L 6e2f1b3101 feat: enhance CivArchive resilience and mirror display
backend:
- add 429 retry handling with backoff in CivArchive API requests
- add page-scraping fallback for CivArchive model payloads
- normalize CivArchive mirrors with dead/gated/paid/sha256 metadata
- include file metadata and mirror data in normalized archive files
- propagate additional dead/missing status field aliases
- include metadata in normalized CivitAI files

frontend:
- render mirror lists with source grouping, dead-link indicators, and skull icons
- add file badges (primary, type, format, size, fp, mirror count, id)
- improve secondary info in resolve/download methods to include base model
- enhance search panel secondary info with base model and filename
- on selecting a version with empty files, fetch fresh version details
- propagate base model context to CivitAI search state and display labels
2026-06-08 03:03:53 +02:00
Dariusz L 65b4594d19 feat: add model details modal and source-aware context menus 2026-06-08 00:53:39 +02:00
Dariusz L 87fabdd9d9 feat: add image preview modal and extended CivitAI image metadata
Adds a fullscreen image preview dialog to the model info panel with
navigation, copy-prompt support, generation metadata display, and
referenced resources listing.

Extends CivitAI backend extraction to handle varied metadata key names
(negativePrompt, Clip skip, Model, etc.) and surfaces clip_skip, width,
height, tags, resources, and raw metadata through the new preview UI.

Renames z-index variable to reserve a dedicated layer for the preview
backdrop.
2026-06-07 18:57:03 +02:00
Dariusz L 420b7e50c2 feat: plumb lora strength and active flags through workflow analysis for accurate model resolution 2026-06-07 17:52:28 +02:00