48 Commits
Author SHA1 Message Date
Dariusz L 63050819fe fix: match workflow hashes by model path instead of widget position 2026-09-10 02:12:36 +02:00
Dariusz L 27ce316c67 perf: optimize hash cache building with in-memory dir entries and thread pool 2026-09-08 18:43:51 +02:00
Dariusz L ddc5333bd2 refactor: unify model data contracts 2026-08-21 23:10:26 +02:00
Dariusz L cfeb333546 refactor: centralize model cache invalidation 2026-08-08 19:03:16 +02:00
Dariusz L 5bfa698282 Unify path identity fallbacks 2026-08-07 15:58:28 +02:00
Dariusz L 233731acce refactor: unify local hash match enrichment 2026-08-06 20:35:38 +02:00
Dariusz L a244369c97 refactor shared path comparison keys 2026-08-05 17:46:06 +02:00
Dariusz L b118753a1e Unify workflow node traversal 2026-08-05 06:00:24 +02:00
Dariusz L 0f70d85f56 Add analysis_id support across workflow analysis pipeline 2026-08-03 20:01:02 +02:00
Dariusz L c294d9d389 Unify local match deduplication 2026-08-03 14:05:05 +02:00
Dariusz L 413e7cc0ab Promote subgraph widget contexts to top-level model references
When a subgraph exposes a promoted widget and the instance value or connected
inner widget contains a usable model path, create a top-level model reference
on the subgraph node instead of only pointing to the inner definition node.
This lets resolution, location, and queue updates target the instance widget
directly.

Also handle serialized subgraph instances that omit proxyWidgets by
reconstructing the same promotion from subgraph input/link metadata, and
extend widget values for promoted inputs in the workflow updater.
2026-08-03 04:17:49 +02:00
Dariusz L 9a7b16cde2 refactor: move missing workflow grouping 2026-08-02 19:40:10 +02:00
Dariusz L c462db3345 refactor: extract workflow inventory 2026-08-02 19:38:35 +02:00
Dariusz L e3fdf9f334 refactor: extract workflow references 2026-08-02 19:28:33 +02:00
Dariusz L b304399798 refactor: slim downloader facade 2026-08-02 18:26:49 +02:00
Dariusz L 1e7cad595c refactor: extract custom-node model adapters from inline handlers 2026-07-28 14:43:37 +02:00
Dariusz L 36fa574562 Preserve workflow UI state and avoid redundant matching during refresh
- Skip fuzzy matching for already-resolved model references in the backend
- Keep existing nodes in their previous result positions during incremental workflow analysis
- Simplify frontend workflow sync and add a content-preserving refresh path
- Preserve scroll position and suppress progress overlay during background Missing/Loaded Models refresh
- Add backend and frontend tests for the new incremental refresh behavior
2026-07-27 18:02:10 +02:00
Dariusz L 15e0b932e8 Add shared workflow model inventory cache and improve progress/hash behavior 2026-07-27 17:05:17 +02:00
Dariusz L 31ad1cde7c refactor: introduce Model Resolver-owned metadata sidecars and build modes 2026-07-27 15:00:18 +02:00
Dariusz L b302566542 refactor: remove dead code and add clear_completed_downloads endpoint 2026-07-18 12:57:29 +02:00
Dariusz L 38dc3e4474 refactor: centralize shared utilities, normalize imports, and configure project linting 2026-07-16 10:52:06 +02:00
Dariusz L 04c3de9533 refactor: reuse shared path and URL normalization utilities
Replace inline regex path/URL normalization in resolver.py and
missing_browser_methods.js with the existing shared helpers:

- core/resolver.py: normalize_workflow_download_url now uses
  prepare_remote_size_probe_url from type_utils instead of a local
  huggingface-specific regex substitution.
- web/resolver/views/missing_browser_methods.js: path identity
  generation now uses normalizePathIdentity from html_utils.js rather
  than ad hoc replace/trim/lowercase logic.

This keeps normalization behavior consistent across the codebase and
removes duplicated regex patterns.
2026-07-16 10:27:06 +02:00
Dariusz L f4a049eca9 feat(core, web): embed workflow model hash metadata and prefer hash-based matching 2026-07-06 18:09:31 +02:00
Dariusz L f691af6387 annotate active downloads in local matches 2026-07-05 05:16:20 +02:00
Dariusz L cbf91eb960 fix: filter local hash match candidates and avoid sidecar self-match 2026-07-03 20:20:36 +02:00
Dariusz L 65ff2baa30 Add local hash match cache and invalidate it on rescan/downloads 2026-07-03 17:09:35 +02:00
Dariusz L 8ce70c30e6 refactor: consolidate shared utilities into type_utils and path_utils
Replace duplicated SHA256 normalization, metadata sidecar discovery, trained-words
extraction, type coercion, and cache helpers into shared modules. Clarify that the
_civitai.py/_civarchive.py/_huggingface.py/_workflow_analyzer.py refactorings are
behavior-preserving so future contributors keep behavior aligned in one place.
2026-07-02 22:41:56 +02:00
Dariusz L 3865914d0f refactor: replace inline JSON reads with read_json_safe in resolver, civitai, and huggingface 2026-07-02 19:19:05 +02:00
Dariusz L 37383ac046 refactor: consolidate logger imports and replace deprecated log.warn usage 2026-07-02 17:27:54 +02:00
Dariusz L 9951e1fbd5 refactor: centralize shared utilities in core/type_utils and remove redundant wrappers
- Move URL model-file validation logic to type_utils.looks_like_model_file and use it from __init__.py, civarchive
- Extract remote file-size probing into type_utils.fetch_remote_file_size and replace duplicated HEAD+Range logic in civarchive and huggingface
- Extract image normalization into type_utils.normalize_model_image, replacing duplicated CivitAI/CivArchive image mapping in civarchive and civitai
- Drop redundant local wrapper functions (_as_dict, _as_list, _version_sort_key, _select_primary_model_file, _extract_primary_file, _read_json_file, _write_json_file_atomic, _get_sha256_hash, _extract_civitai_image_id, _build_civitai_image_url) in favor of existing shared implementations
- Use path_utils.read_json_safe and write_json_atomic directly in model_list
2026-07-02 04:40:23 +02:00
Dariusz L 86bd6c04fc refactor: centralize model-extension handling and search-cache clearing
- Add clear_all_search_caches() in core/sources/__init__.py and replace 4
  individual cache-clearing call sites in __init__.py with the new helper.
- Add shared parse_civitai_model_path() in core/type_utils.py and use it
  from both CivitAI and CivArchive URL parsers to remove duplicated regex logic.
- Rename parse_huggingface_url() to get_huggingface_repo_and_file_from_url()
  in core/resolver.py for clarity.
- Add stripModelExtension() and hasModelExtension() utilities (with a single
  MODEL_EXTENSIONS_REGEX) in web/resolver/utils/render_format_methods.js
  and replace repeated inline regex patterns across search_panel.js,
  model_info_methods.js, tabs_loaded_methods.js, and queue_methods.js.
2026-07-02 03:57:43 +02:00
Dariusz L 2d2e8b4621 refactor: centralize filename and size utilities across sources
- Introduce get_filename_from_path in path_utils and replace repeated
  os.path.basename calls across core/downloader.py, core/resolver.py,
  core/sources/civarchive.py, core/sources/civitai.py, and
  core/sources/huggingface.py
- Move duplicated inline HTTP response size helpers (_response_file_size,
  _parse_size_header, _parse_content_range_size) into core/type_utils.py
  and update civarchive.py and huggingface.py to import them
- Remove inline normalize_sha256_value from __init__.py and reuse the
  existing normalize_sha256 helper
- Refactor options_methods.js localStorage read/write logic into a
  SETTINGS_MAP-driven loop to reduce repetition
2026-07-02 03:37:44 +02:00
Dariusz L 1128234bbd feat: add local model hash comparison functionality
Implement end-to-end hash comparison for local models and local matches.

Backend:
- Add /model_resolver/local-model-hashes API endpoint
- Add get_local_model_hash_metadata() to read sidecar SHA256 metadata

Frontend:
- Add Compare Hashes context menu item and CSS
- Pass missing_key/missing_search_key through resolveDownloadMethods
- Enhance local match rendering with hash status and context data
- Add hash normalization, formatting, and candidate collection helpers
- Wire compareLocalModelHashesWithCurrentFinding action

Also update HuggingFace author index metadata.
2026-07-02 00:27:59 +02:00
Dariusz L a7f5f70e3c refactor: migrate logging from global functions to per-module loggers
Replace global log_* wrapper functions with create_module_logger(__name__)
instances across the codebase, enabling proper module-level logger
namespacing. Simplifies core/log_system/log_funcs.py by removing legacy
_normalize_module_path and cleaning up _normalize_module_name. Bumps
log_system config to 2.0.0 and introduces PROJECT_ALIASES for cleaner
log module prefix stripping.
2026-06-30 21:24:33 +02:00
Dariusz L d979bbcf21 refactor: extract shared utilities and deduplicate metadata path definitions
- Add first_non_empty() and to_int() to core/type_utils.py
- Add get_filename_from_path() and METADATA_DIR to core/path_utils.py
- Replace inline implementations across resolver, downloader, and sources
with calls to the shared utilities
- Move strip_known_model_extension from resolver to matcher
2026-06-25 17:00:27 +02:00
Dariusz L 905faa9056 refactor: centralize MODEL_EXTENSIONS in type_utils to eliminate duplication 2026-06-25 16:53:32 +02:00
Dariusz L e4c3b0fb6f refactor: consolidate type and progress utilities
Add core/type_utils.py with as_dict/as_list helpers, and
core/progress.py with a unified report_progress() callback.

Replace inline duplicated implementations of _json_safe_metadata(),
_as_list(), normalize_filename(), and per-source _report_progress()
with calls to the shared modules across downloader, matcher, resolver,
and all source modules (CivitAI, CivArchive, HuggingFace, LoRA Manager).
Extend filename normalization to include dots as separators.
2026-06-25 02:09:47 +02:00
Dariusz L 8f890f6992 refactor: centralize path utilities in core/path_utils
Extract repeated path normalization and identity logic from __init__.py,
core/resolver.py, core/downloader.py, and core/scanner.py into a shared
core/path_utils module. Also add a loading state to the local model picker
so it waits for models to load before rendering options.
2026-06-25 01:23:20 +02:00
Dariusz L 34638d517e refactor: share normalize_sha256 and parse_huggingface_url across modules 2026-06-25 00:03:34 +02:00
Dariusz L 5445dd4ac0 feat: implement local hash matching via metadata sidecar files
Backend adds SHA256 normalization, metadata sidecar lookup, and hash fields across source modules. Frontend handles local_hash_matches in search responses and missing browser.
2026-06-24 21:59:10 +02:00
Dariusz L 8c0168ab76 feat: skip model analysis for empty or non-model workflows
Add early-return optimizations so both backend and frontend avoid unnecessary
URL extraction and disk scanning when a workflow has no nodes or contains
no potential model references.

Also tighten API input validation by rejecting non-object workflow_json.
2026-06-24 03:46:50 +02:00
Dariusz L b086ffd106 Improve symlink/junction handling and add RapidFuzz for faster fuzzy matching
- Add rapidfuzz as optional dependency with SequenceMatcher fallback
- Use stable path identity (realpath) for deduplication in scanner and resolver
- Prevent duplicate scans and duplicate model entries across symlinks/junctions
- Improve matching performance with RapidFuzz when available
2026-06-22 02:36:40 +02:00
Dariusz L a499af3345 Enable local model matching for resolved models and prevent symlink/junction scan loops 2026-06-22 02:21:24 +02:00
Dariusz L 590b4a4969 feat(missing-browser): add toggle to show resolved models
Report resolved (already-existing) models alongside missing ones in the
missing-browser view. Resolved models are hidden by default; a new
'Show resolved' toggle in the toolbar reveals them with a badge and
reduced-opacity row styling. The toggle state is persisted in
localStorage.

Backend: include resolved_models and total_resolved in the analysis
payload so the frontend can merge them into the display list.

Frontend: normalize resolved workflow models into the missing-models
display shape, compute visibility-aware counts, and refresh the
missing-browser view on queue mutating actions.
2026-06-22 01:34:55 +02:00
Dariusz L 4f91accbab feat: improve model scan cache override, combo UX, and filename handling
- Add force_rescan end-to-end so callers can bypass the short-lived local model scan cache: backend entry in __init__.py and core/resolver.py, lifecycle_graph_methods forwards it through the workflow analysis body, and download_target_methods passes ?force=1 from the refresh flow.
- Rewrite combo dropdown lifecycle in missing_browser_methods with show/hide helpers, pointer-capture tracking, and a capture-phase outside-click listener so the list stays open during drag/hover and closes reliably otherwise.
- Animate refresh buttons (CSS keyframes + JS Web Animations) with reduced-motion fallback.
- Encode selected model path into the selected-bar data-model attribute and wire contextmenu so users can open the containing folder from the selection.
- Fix XSS risk in selected-bar label by escaping label HTML; change queue/selected Remove buttons to danger style.
- Increase max missing-browser model column width clamp from 430 to 820 and stop truncating missing-filename display; show full filename with tooltip instead.
- Update huggingface-author-index.json metadata timestamp/counts (1 file changed, 1 line).
2026-06-19 03:57:40 +02:00
Dariusz L c9dd0e47ac feat: improve model resolution for extensionless workflow values and add new node type support
- Add model_title match type for CivArchive and CivitAI sources, resolving
  extensionless workflow values by title similarity (threshold 82.0)
- Recognize new node types and widget indices across model categories
  (GGUF, CLIP, ControlNet, StyleModel, Hypernetwork, etc.)
- Scan known model combo widget indices even when the stored value has no
  file extension, treating non-placeholder strings as model references
- Strip known model extensions during workflow URL lookup so filename stems
  match regardless of extension
- Update HuggingFace author index metadata
2026-06-18 02:19:14 +02:00
Dariusz L 541881ce49 refactor: replace post-download re-analysis with targeted local match refresh
Replace the heavy autoResolveAfterDownload path (which re-analyzed the
entire workflow) with a more efficient refreshLocalMatchesForDownloadedMissing
call that queries /model_resolver/local-matches directly for the
specific downloaded file.

This removes the unnecessary re-analysis step, adds an explicit
force_rescan flag (coerced to bool on the backend for safe JSON
payload handling), propagates the missing model's category through
the activeDownloads tracking objects, and updates related UI text
to reflect the new narrower behavior.
2026-06-06 18:09:06 +02:00
Dariusz L 5a8e68655f refactor: rename Model Linker to Model Resolver
Replace all link dialog terminology with resolver terminology
throughout the codebase, including module names, CSS classes,
IDs, JS variables, SVG gradient IDs, and log module names.
Remove the old web/linker/ tree and introduce the new web/resolver/
tree with parallel structure. Also replace core/linker.py with
core/resolver.py.
2026-06-05 05:53:39 +02:00