Commit Graph
8 Commits
Author SHA1 Message Date
Dariusz L ed571a8ec6 feat: classify CivArchive failures 2026-08-02 21:57:46 +02:00
Dariusz L 097be623f4 feat: preserve CivArchive page URLs separate from HuggingFace mirrors, fix HTML array escaping, and accept provider array payloads
Backend:
- Keep CivArchive source page URL distinct from HuggingFace download mirrors in download metadata
- Handle CivitAI hash lookup HTTP errors and unexpected JSON shapes without caching failures
- Allow network JSON requests to return arrays as well as objects

Frontend:
- Escape each item individually when interpolating arrays into HTML templates
- Prefer the selected provider's page URL over cross-provider mirrors in context menus and download targets
- Reuse source link resolution for model info display

Tests:
- Add coverage for CivArchive metadata URL isolation, HTML array escaping, provider array responses, and CivitAI hash lookup caching behavior
2026-07-20 16:16:09 +02:00
Dariusz L 0b53fca8c6 refactor: unify category resolution, network requests, sidecar lookup, and progress tracking
Consolidate provider-specific category mapping into resolve_model_category(),
shared JSON request handling into execute_provider_json_request(), sidecar
path resolution into get_or_build_model_sidecar_info(), and progress
reporting into ProgressTracker. Update civitai/civarchive/common sources to
use the unified helpers. Add tests for the new utilities. Remove redundant
frontend wrapper methods from render_format_methods.js.
2026-07-18 12:48:11 +02:00
Dariusz L 759aef99f1 Unify model result building and progress payload updates 2026-07-18 12:26:17 +02:00
Dariusz L 38dc3e4474 refactor: centralize shared utilities, normalize imports, and configure project linting 2026-07-16 10:52:06 +02:00
Dariusz L 8cf688eba6 refactor: centralize source API requests, polling, and path parsing
- Add request_source_response and request_source_json helpers with 429 retry logic
- Introduce shared split_path_segments and parse_provider_model_url utilities
- Migrate CivitAI, CivArchive, HuggingFace, and ModelList sources to centralized helpers
- Add generic pollBackgroundTask utility for frontend progress polling
- Add tests for request_source_json, split_path_segments, parse_provider_model_url
- Add huggingface-hub to requirements
2026-07-14 12:03:35 +02:00
Dariusz L 1f0edaaa7e Harden aria2 downloads, retry RPC, and preserve HuggingFace XET redirect headers 2026-07-14 02:33:44 +02:00
Dariusz L 48db08ae1a Harden network requests, downloads, and path handling
- Add core/network_utils.py with URL validation, public-IP enforcement,
  redirect validation, and safe request_public_url/validate_public_http_url helpers
- Replace direct requests calls in downloader/type_utils/sources with
  request_public_url so redirects are validated before follow-up
- Replace netloc string checks with host_matches_domain for CivitAI,
  CivArchive, and HuggingFace host matching
- Add HuggingFace Xet download backend with progress reporting and
  cancellation support via hf_xet/huggingface_hub
- Harden aria2 installer: reject symlinks/special files in archives,
  restrict custom aria2c paths to the managed install, and update UI copy
- Normalize paths with realpath and enforce configured model roots in
  local hash/metadata routes, plus remove caller-selected metadata_path
  injection in sidecar writing
- Sanitize download/network errors to strip signed query strings before
  logging or showing them to the user
2026-07-14 01:33:19 +02:00