Commit Graph
19 Commits
Author SHA1 Message Date
Vito Sansevero efcbd5a8aa fix: gate the LoraManager serving roots and accept relative link paths
A LoraManager path hand-edited into config.json went straight into
find_lora_directories and so into the image-serving allow-list. When a
path is configured it must now pass lora_utils.resolve_lora_manager_path
(a custom_nodes child whose name contains 'lora'); otherwise, or when the
resolver is unavailable, no LoRA directories are served. An empty path
keeps the custom_nodes-only auto-detection.

POST /images/link now also accepts the relative forms the API hands out
(root-relative 'sub/a.png' and ComfyUI-relative 'output/sub/a.png'),
resolving them against the allowed roots and anchors; '..' segments in
a relative path are refused and unknown files are 404.
2026-10-08 18:01:34 -07:00
Vito Sansevero cd209131ee fix: check thumbnail directory containment with a real path test
clear_thumbnails compared resolved paths with str.startswith, so a
thumbnails directory that resolved to a sibling with a shared prefix
(out vs out2) passed the safety check and was emptied. It now uses the
module's _is_within (realpath + normcase + Path.is_relative_to).
2026-10-08 18:01:34 -07:00
Vito Sansevero e720aeb73a fix: describe images relative to the ComfyUI tree in API responses
Gallery and prompt responses carried absolute server paths: /images/output
entries (path, root_dir), thumbnails_path, the images/prompt lookup, every
image list (image_path plus its normalised file_path duplicate) and the
preview images attached to prompt listings. They now go through
_public_path: /images/output 'path' is relative to its root (what the
delete and autotag endpoints resolve against), 'root_dir' and every
other image_path are relative to the ComfyUI base, and file_path is
dropped. recent/all/search image lists also gain url/thumbnail_url via
the same enrichment the prompt-images route already had. Remaining
str(e) error bodies in both modules use the basename-only error text.
2026-10-08 18:01:34 -07:00
Vito Sansevero 1c37f0ab56 fix: make the thumbnail progress stream POST-only
GET /prompt_manager/images/generate-thumbnails/progress wrote thumbnails
to disk, so any prefetch, link preview or <img src> could start a full
output-directory scan. The route now accepts POST only (GET answers
405); quality still comes from the query string so the SSE client can
keep the same URL shape.
2026-10-08 18:01:34 -07:00
Vito Sansevero 1f422d2c43 fix: restore through SQLite's backup API, close_all for shared files, safer output scans
restore_from_file copied the uploaded file over the live database and deleted
its WAL while other PromptModel instances (nodes, image monitor, queue hook)
still held connections, which corrupts on POSIX and fails on Windows. It now
copies pages through the online backup API into the live connection so every
open handle sees the new content. PromptDatabase.close_all() closes the
connections of every thread, which tests need before deleting a database.
Output scans skip symlinked files, filesystem entries get a stable digest id
instead of a per-process hash(), and thumbnail streaming stops when the
client disconnects.
2026-10-08 18:01:34 -07:00
Vito Sansevero 0393d01746 fix: page image listings in SQL and drop three unused helpers
get_recent_images and search_images_by_prompt take limit/offset so the recent
and search image routes no longer over-fetch a 5000-row window and slice it.
generate_content_hash, PromptModel.migrate_database and find_duplicates had no
callers outside their own tests. The isolated logger tests re-apply their
thread filter after update_config rebuilds the handlers. Long lines in the
touched database and image modules are wrapped so the full flake8 hook passes.
2026-10-08 18:01:34 -07:00
Vito Sansevero 027fdc09bb fix: cap every request-driven resource in the image routes
- /images/recent, /images/all and /images/output parse limit/offset
  through parse_page_params on every path (absent, 0, negative, float
  strings), and the clamped values are what reach the DB. /images/all
  gets its own MAX_BULK_LIMIT ceiling instead of an unbounded query;
  /images/recent caps its fetch window at MAX_RECENT_IMAGES_WINDOW.
- parse_page_params also clamps offset to MAX_PAGE_OFFSET so values past
  int64 no longer surface as a 500 from SQLite.
- Gallery and thumbnail scans share _iter_media_files: no symlink
  following, MAX_SCAN_DEPTH, and MAX_GALLERY_FILES / MAX_THUMBNAIL_FILES
  caps. Thumbnail size comes only from named presets.
- JSON bodies on /images/link and /images/generate-thumbnails are read
  in bounded chunks; over MAX_JSON_BODY_BYTES answers 413, malformed or
  non-object JSON answers 400.
- /images/link only accepts media paths inside the allowed directories
  (403 otherwise), closing a path-existence oracle.
2026-10-08 18:01:34 -07:00
Vito Sansevero 5fddf56eee perf: thumbnail generation no longer blocks the event loop
generate_thumbnails_with_progress is split into _thumbnail_targets
(scan, executor), _generate_one (one PIL/ffmpeg job, executor) and an
async driver that awaits each file via _run_in_executor, emits one
progress frame per file and yields between files. Per-file failures
are reported as file_error events (error stays fatal for the UI) and
the loop continues. The wildcard Access-Control-Allow-Origin header is
gone. The blocking POST variant reuses the same helpers. The scan never
follows symlinks and is capped by MAX_SCAN_DEPTH / MAX_THUMBNAIL_FILES.
2026-10-08 18:01:34 -07:00
Vito Sansevero 3228824161 fix: bound page sizes on list endpoints
/search, /recent and /images/recent share parse_page_params: limit is
clamped to [1, MAX_PAGE_LIMIT=500], offset to >= 0, and non-integer
values answer 400 instead of 500. /search now honours offset and all
three return a pagination block with the effective limit and offset.
2026-10-08 18:01:34 -07:00
Vito Sansevero 191d02eb6c fix: image routes serve only media files inside allowed directories
serve_image and serve_output_image now reject anything whose extension
is not in IMAGE_EXTENSIONS, fail closed when no output directory is
configured, and answer 403 (not 404) for dot-dot, absolute and
symlink-escaping paths. Containment is checked after realpath+normcase
so it behaves the same on Windows, macOS and Linux. 500 responses use
safe_error_message so OSError text no longer leaks absolute paths.
2026-10-08 18:01:34 -07:00
c01d8e3c12 feat: recursive subfolder filtering + gallery tree navigation (#140)
* feat: recursive subfolder filtering + gallery tree navigation + thumbnail sizing

- Recursive subfolder filtering: selecting an intermediate folder shows all child images
- Gallery tree navigation panel with collapsible folder hierarchy (sticky, resizable)
- Thumbnail size controls (S/M/L) for adjustable grid density
- Fix path collision when multiple gallery roots share overlapping relative paths
- Fix auto-detect mode collapsing nested folders to ambiguous basenames
- Security test suite for path traversal edge cases

Addresses feedback from #126. Fixes code review items P1/P2 from #135.

Co-Authored-By: Kai Bowie <kaigtb@proton.me>

* style: fix Black formatting in test_path_security.py

* feat(gallery): add folder tree sidebar with hierarchical navigation (#126)

- Add collapsible folder tree sidebar to gallery view
- Support hierarchical year/month/date folder navigation
- Include ancestor path expansion for proper tree structure
- Add subfolder filtering to gallery image queries
- Update admin subfolder dropdown with ancestor support
- Fix trailing whitespace in SQL queries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kai <noreply@gtb.ai>
Co-authored-by: Kai Bowie <kaigtb@proton.me>
Co-authored-by: Kai <kai@tachi-02.localdomain>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-27 07:58:59 -07:00
Vito 00284d92da feat: LoraManager integration — trigger words & example import (#52) (#136)
* feat: add LoRA utility module for LoraManager integration (#52)

Detection, metadata reading, trigger word cache, and prompt injection
for ComfyUI-Lora-Manager. All functions return empty results when
LoraManager is not installed.

* feat: add IntegrationConfig for opt-in third-party integrations (#52)

* feat(api): add LoraManager integration endpoints (#52)

Detection, enable/disable, scan/import, trigger word lookup,
and cache refresh endpoints under /prompt_manager/lora/*.

* feat: inject LoRA trigger words into prompts at encoding time (#52)

When enabled, scans for <lora:NAME:WEIGHT> tags in prompt text and
appends trigger words from LoraManager metadata. Behind config toggle.

* feat(ui): add LoraManager integration settings and import UI (#52)

Integrations section in settings with auto-detection badge, enable
toggle, trigger word toggle, and Import LoRA Data button.

* fix: detect LoraManager with case-insensitive directory scan (#52)

* fix: simplify LoraManager heuristic to match real install structure (#52)

* fix: find ComfyUI root via folder_paths and handle symlinked installs (#52)

* fix(ui): add pointer-events-none to toggle switch overlays (#52)

The styled div was intercepting clicks meant for the sr-only checkbox
input, preventing toggle switches from being clickable.

* fix: use lora-manager category and tag for imported LoRA data (#52)

* fix: use rglob to find metadata in lora subdirectories (#52)

* feat: discover LoRA dirs from extra_model_paths.yaml and folder_paths (#52)

Only models/loras under the ComfyUI root was checked. Now also parses
extra_model_paths.yaml and uses folder_paths.get_folder_paths('loras')
at runtime to find all configured LoRA directories.

* fix: use example prompts instead of trigger words, serve LoRA preview images (#52)

- Prompt text now uses civitai example prompts (images[].meta.prompt)
  when available, falling back to model name instead of trigger words
- Image serving now allows paths within LoRA directories when the
  integration is enabled, fixing 403 errors on preview images

* fix: handle null civitai field, link all preview images (#52)

- Guard against civitai: null in metadata (was crashing the scan)
- Link all local preview images per LoRA, not just the first
- Add get_civitai_image_urls() for future remote image support

* feat: reimport clears previous lora-manager data first (#52)

Clicking Import LoRA Data now deletes all existing lora-manager
category prompts before scanning, ensuring a clean reimport.

* feat: download civitai example images during LoRA import (#52)

Images from civitai.images[] are downloaded to data/lora_images/ cache
and linked to prompts alongside local preview files. Cached files are
reused on reimport. Image serving allows the cache directory.

* feat: add CivitAI API key setting for authenticated image downloads (#52)

Most civitai example images (especially NSFW) require authentication.
Adds API key field to the Integrations settings panel, passed as
Bearer token when downloading example images.

* feat(ui): add progress modal for LoRA import (#52)

Closes settings modal and shows a dedicated progress modal with
progress bar, status text, and processed/imported counts during
LoRA import. Auto-closes after completion.

* fix: reduce download timeout to 5s, update progress per LoRA (#52)

5K images at 15s timeout was painfully slow. Reduced to 5s fail-fast.
Progress now updates for every LoRA with image count, not every 5th.

* perf: use 512px thumbnails and parallel downloads for LoRA images (#52)

Full-size civitai images averaged 5.6MB each (27GB total for 5K images).
Now requests /width=512/ thumbnails (~50-100KB) and downloads 8 in
parallel per LoRA. Expected speedup: ~100x smaller + 8x parallel.

* fix: resize downloaded images to 512px thumbnails locally (#52)

Civitai CDN returns 401 for /width=N/ thumbnail URLs with API key auth.
Instead, download the original and resize to 512px via PIL before saving.
Reduces disk usage from ~5MB to ~30-50KB per image.

* docs: update README for v3.2.1 LoRA Manager integration (#52)

- Add LoRA Manager integration section with setup guide and CivitAI key docs
- Add folder filter section with rescan note for existing libraries
- Add v3.2.1 changelog entry, split from v3.2.0
- Add WIP notice for LoRA Manager feature
- Update AutoTag section to include WD14 models
- Fix stale references (KikoTextEncode, outdated file structure)
- Remove dated v2 development note
- Add screenshots for settings, integration, and filtered results
- Fix code review items: remove unused constant, add comments to empty excepts
- Bump version to 3.2.1

* test: add unit tests for LoRA integration and folder filter (#52)

- test_lora_utils.py: 32 tests covering trigger word extraction, example
  prompts, image URLs, metadata parsing, cache dir, TriggerWordCache
- test_lora_database.py: 17 tests covering delete_prompts_by_category,
  folder filter search, get_prompt_subfolders, LoRA import workflow
- test_config.py: 7 new IntegrationConfig tests for structure, enable,
  partial update, reset, and roundtrip

* fix: remove unused MagicMock import in test_lora_utils (#52)
2026-04-03 18:26:23 -07:00
Vito 6fc6e683eb feat: multi-directory gallery scan with folder filter (#126) (#135)
* refactor(api): add multi-root output dir infrastructure (#126)

- Change _gallery_cache from None to {} (dict keyed by path string)
- Update invalidate_gallery_cache() to clear dict instead of setting None
- Add _get_all_output_dirs() helper as single source of truth for
  resolving configured + fallback output directories
- Update _enrich_prompt_images() to iterate all output roots when
  computing relative paths and thumbnail URLs

* feat(api): support multiple gallery scan paths in settings endpoint (#126)

* feat(ui): dynamic multi-path settings with add/remove buttons (#126)

* cleanup: remove dead galleryRootPath assignment in loadSettings

* feat(api): multi-directory gallery scan with subfolder filter and multi-root serve (#126)

* feat(db): add folder filter to search_prompts and get_prompt_subfolders (#126)

* feat(api): add subfolders endpoint and folder filter to prompt search (#126)

* feat(ui): add folder dropdown filter to search bar (#126)

* fix: escape LIKE wildcards in folder filter and remove dead _gallery_cache_time field

* feat(scan): scan all media types across all output dirs with batched extraction (#126)

* chore: gitignore SQLite WAL-mode temp files

* fix: enrich prompt images with URLs and check all output dirs for serving

- Extract _enrich_images() for flat image lists, used by get_prompt_images
- get_prompt_images now adds url/thumbnail_url (fixes stuck "Loading images...")
- serve_image checks all output dirs not just first (fixes 403 on multi-path)
- Fix diagnostics monitor check to read current module attribute, not stale import

* feat(ui): add version-scoped update notice popup for rescan (#126)

* chore: bump version to 3.2.0
2026-04-03 06:46:01 -07:00
Vito Sansevero 7f92c4d5b8 fix: preserve full tracebacks in server-side error logging
Use logger.exception() and exc_info=True to retain stack traces
for debugging, addressing Codex review feedback on PR #110.
2026-02-08 05:33:06 -08:00
Vito Sansevero be205f3dcc feat: add pre-commit hooks for local CI checks
- Black (auto-fix formatting on commit)
- Flake8 (block on syntax errors and undefined names)
- Bandit (security scan, non-blocking to match CI)
- Tests (run on pre-push only)
- Apply Black formatting fixes caught by the new hooks
2026-02-08 05:27:03 -08:00
Vito Sansevero ed40be066c fix: prevent stack trace exposure in SSE error responses
Log exception details server-side only; send generic error messages
to clients via SSE streams to resolve code scanning alerts.
2026-02-08 05:23:04 -08:00
Vito Sansevero 113d663fce fix: resolve 15 GitHub code scanning security alerts
- Fix XSS in showFullPrompt() across admin.js, gallery.js, metadata.html
  by replacing HTML string interpolation with DOM manipulation (createElement
  + textContent), eliminating unsafe user content injection
- Fix stack trace exposure in py/api error responses by replacing str(e)
  with generic messages and adding logger.exception() for server-side
  traceability
2026-02-07 13:04:52 -08:00
Vito Sansevero 2737a4b1ef style: apply Black formatter (line-length=88) to all Python files
Automated formatting pass across 30 files to establish consistent code
style enforced by CI. No logic changes.
2026-02-07 08:30:09 -08:00
Vito Sansevero 4548beb723 refactor: split api.py into domain modules and extract frontend JS
Phase 4 structural refactoring:
- Split monolithic py/api.py (5.3k lines) into domain mixins:
  prompts.py, images.py, admin.py, logging_routes.py, autotag_routes.py
- Extract inline JS from admin.html into web/js/admin.js
- Extract inline JS from gallery.html into web/js/gallery.js
- Add gzip compression middleware to API responses (Phase 5.4)
- Standardize API error envelope with success: false (Phase 5.3)
- Add filmstrip image enrichment to prompt list responses (Phase 5.2)
2026-02-07 07:02:25 -08:00