* feat: add LoRA utility module for LoraManager integration (#52) Detection, metadata reading, trigger word cache, and prompt injection for ComfyUI-Lora-Manager. All functions return empty results when LoraManager is not installed. * feat: add IntegrationConfig for opt-in third-party integrations (#52) * feat(api): add LoraManager integration endpoints (#52) Detection, enable/disable, scan/import, trigger word lookup, and cache refresh endpoints under /prompt_manager/lora/*. * feat: inject LoRA trigger words into prompts at encoding time (#52) When enabled, scans for <lora:NAME:WEIGHT> tags in prompt text and appends trigger words from LoraManager metadata. Behind config toggle. * feat(ui): add LoraManager integration settings and import UI (#52) Integrations section in settings with auto-detection badge, enable toggle, trigger word toggle, and Import LoRA Data button. * fix: detect LoraManager with case-insensitive directory scan (#52) * fix: simplify LoraManager heuristic to match real install structure (#52) * fix: find ComfyUI root via folder_paths and handle symlinked installs (#52) * fix(ui): add pointer-events-none to toggle switch overlays (#52) The styled div was intercepting clicks meant for the sr-only checkbox input, preventing toggle switches from being clickable. * fix: use lora-manager category and tag for imported LoRA data (#52) * fix: use rglob to find metadata in lora subdirectories (#52) * feat: discover LoRA dirs from extra_model_paths.yaml and folder_paths (#52) Only models/loras under the ComfyUI root was checked. Now also parses extra_model_paths.yaml and uses folder_paths.get_folder_paths('loras') at runtime to find all configured LoRA directories. * fix: use example prompts instead of trigger words, serve LoRA preview images (#52) - Prompt text now uses civitai example prompts (images[].meta.prompt) when available, falling back to model name instead of trigger words - Image serving now allows paths within LoRA directories when the integration is enabled, fixing 403 errors on preview images * fix: handle null civitai field, link all preview images (#52) - Guard against civitai: null in metadata (was crashing the scan) - Link all local preview images per LoRA, not just the first - Add get_civitai_image_urls() for future remote image support * feat: reimport clears previous lora-manager data first (#52) Clicking Import LoRA Data now deletes all existing lora-manager category prompts before scanning, ensuring a clean reimport. * feat: download civitai example images during LoRA import (#52) Images from civitai.images[] are downloaded to data/lora_images/ cache and linked to prompts alongside local preview files. Cached files are reused on reimport. Image serving allows the cache directory. * feat: add CivitAI API key setting for authenticated image downloads (#52) Most civitai example images (especially NSFW) require authentication. Adds API key field to the Integrations settings panel, passed as Bearer token when downloading example images. * feat(ui): add progress modal for LoRA import (#52) Closes settings modal and shows a dedicated progress modal with progress bar, status text, and processed/imported counts during LoRA import. Auto-closes after completion. * fix: reduce download timeout to 5s, update progress per LoRA (#52) 5K images at 15s timeout was painfully slow. Reduced to 5s fail-fast. Progress now updates for every LoRA with image count, not every 5th. * perf: use 512px thumbnails and parallel downloads for LoRA images (#52) Full-size civitai images averaged 5.6MB each (27GB total for 5K images). Now requests /width=512/ thumbnails (~50-100KB) and downloads 8 in parallel per LoRA. Expected speedup: ~100x smaller + 8x parallel. * fix: resize downloaded images to 512px thumbnails locally (#52) Civitai CDN returns 401 for /width=N/ thumbnail URLs with API key auth. Instead, download the original and resize to 512px via PIL before saving. Reduces disk usage from ~5MB to ~30-50KB per image. * docs: update README for v3.2.1 LoRA Manager integration (#52) - Add LoRA Manager integration section with setup guide and CivitAI key docs - Add folder filter section with rescan note for existing libraries - Add v3.2.1 changelog entry, split from v3.2.0 - Add WIP notice for LoRA Manager feature - Update AutoTag section to include WD14 models - Fix stale references (KikoTextEncode, outdated file structure) - Remove dated v2 development note - Add screenshots for settings, integration, and filtered results - Fix code review items: remove unused constant, add comments to empty excepts - Bump version to 3.2.1 * test: add unit tests for LoRA integration and folder filter (#52) - test_lora_utils.py: 32 tests covering trigger word extraction, example prompts, image URLs, metadata parsing, cache dir, TriggerWordCache - test_lora_database.py: 17 tests covering delete_prompts_by_category, folder filter search, get_prompt_subfolders, LoRA import workflow - test_config.py: 7 new IntegrationConfig tests for structure, enable, partial update, reset, and roundtrip * fix: remove unused MagicMock import in test_lora_utils (#52)
317 lines
11 KiB
Python
317 lines
11 KiB
Python
"""
|
|
Unit tests for LoRA Manager integration utilities.
|
|
|
|
Tests metadata parsing, trigger word extraction, image URL extraction,
|
|
directory detection, TriggerWordCache, and image download logic.
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
import tempfile
|
|
import threading
|
|
import unittest
|
|
from pathlib import Path
|
|
from unittest.mock import patch
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
|
|
from py.lora_utils import (
|
|
TriggerWordCache,
|
|
get_civitai_image_urls,
|
|
get_example_prompt_from_metadata,
|
|
get_lora_image_cache_dir,
|
|
get_trigger_words_from_metadata,
|
|
read_lora_metadata,
|
|
)
|
|
|
|
# ── Sample metadata fixtures ───────────────────────────────────────────
|
|
|
|
|
|
def _make_metadata(
|
|
trained_words=None,
|
|
images=None,
|
|
model_name="test_lora",
|
|
file_name="test.safetensors",
|
|
):
|
|
"""Build a realistic LoRA metadata dict for testing."""
|
|
meta = {"file_name": file_name}
|
|
civitai = {}
|
|
if trained_words is not None:
|
|
civitai["trainedWords"] = trained_words
|
|
if images is not None:
|
|
civitai["images"] = images
|
|
if model_name:
|
|
civitai["model"] = {"name": model_name}
|
|
if civitai:
|
|
meta["civitai"] = civitai
|
|
return meta
|
|
|
|
|
|
# ── Pure function tests (no mocking) ──────────────────────────────────
|
|
|
|
|
|
class TestGetTriggerWords(unittest.TestCase):
|
|
"""Test get_trigger_words_from_metadata — pure dict extraction."""
|
|
|
|
def test_extracts_words(self):
|
|
meta = _make_metadata(trained_words=["word1", "word2", "word3"])
|
|
self.assertEqual(
|
|
get_trigger_words_from_metadata(meta), ["word1", "word2", "word3"]
|
|
)
|
|
|
|
def test_strips_whitespace(self):
|
|
meta = _make_metadata(trained_words=[" padded ", "\ttabbed\t"])
|
|
self.assertEqual(get_trigger_words_from_metadata(meta), ["padded", "tabbed"])
|
|
|
|
def test_filters_empty_strings(self):
|
|
meta = _make_metadata(trained_words=["valid", "", " ", "also_valid"])
|
|
self.assertEqual(get_trigger_words_from_metadata(meta), ["valid", "also_valid"])
|
|
|
|
def test_no_civitai_key(self):
|
|
self.assertEqual(get_trigger_words_from_metadata({}), [])
|
|
|
|
def test_no_trained_words(self):
|
|
meta = _make_metadata()
|
|
self.assertEqual(get_trigger_words_from_metadata(meta), [])
|
|
|
|
def test_trained_words_not_list(self):
|
|
meta = {"civitai": {"trainedWords": "not a list"}}
|
|
self.assertEqual(get_trigger_words_from_metadata(meta), [])
|
|
|
|
def test_non_string_items_filtered(self):
|
|
meta = _make_metadata(trained_words=["valid", 123, None, "also_valid"])
|
|
self.assertEqual(get_trigger_words_from_metadata(meta), ["valid", "also_valid"])
|
|
|
|
|
|
class TestGetExamplePrompt(unittest.TestCase):
|
|
"""Test get_example_prompt_from_metadata — extracts first usable prompt."""
|
|
|
|
def test_extracts_first_prompt(self):
|
|
images = [
|
|
{"meta": {"prompt": "a beautiful landscape"}},
|
|
{"meta": {"prompt": "second prompt"}},
|
|
]
|
|
meta = _make_metadata(images=images)
|
|
self.assertEqual(
|
|
get_example_prompt_from_metadata(meta), "a beautiful landscape"
|
|
)
|
|
|
|
def test_skips_empty_prompts(self):
|
|
images = [
|
|
{"meta": {"prompt": ""}},
|
|
{"meta": {"prompt": " "}},
|
|
{"meta": {"prompt": "valid prompt"}},
|
|
]
|
|
meta = _make_metadata(images=images)
|
|
self.assertEqual(get_example_prompt_from_metadata(meta), "valid prompt")
|
|
|
|
def test_no_images(self):
|
|
meta = _make_metadata(images=[])
|
|
self.assertIsNone(get_example_prompt_from_metadata(meta))
|
|
|
|
def test_no_civitai(self):
|
|
self.assertIsNone(get_example_prompt_from_metadata({}))
|
|
|
|
def test_images_without_meta(self):
|
|
images = [{"url": "http://example.com/img.jpg"}]
|
|
meta = _make_metadata(images=images)
|
|
self.assertIsNone(get_example_prompt_from_metadata(meta))
|
|
|
|
def test_meta_without_prompt(self):
|
|
images = [{"meta": {"seed": 12345}}]
|
|
meta = _make_metadata(images=images)
|
|
self.assertIsNone(get_example_prompt_from_metadata(meta))
|
|
|
|
def test_non_dict_images_skipped(self):
|
|
images = ["not a dict", None, {"meta": {"prompt": "found it"}}]
|
|
meta = _make_metadata(images=images)
|
|
self.assertEqual(get_example_prompt_from_metadata(meta), "found it")
|
|
|
|
def test_non_string_prompt_skipped(self):
|
|
images = [{"meta": {"prompt": 12345}}, {"meta": {"prompt": "real prompt"}}]
|
|
meta = _make_metadata(images=images)
|
|
self.assertEqual(get_example_prompt_from_metadata(meta), "real prompt")
|
|
|
|
|
|
class TestGetCivitaiImageUrls(unittest.TestCase):
|
|
"""Test get_civitai_image_urls — extracts image URLs from metadata."""
|
|
|
|
def test_extracts_urls(self):
|
|
images = [
|
|
{"url": "https://civitai.com/img1.jpg"},
|
|
{"url": "https://civitai.com/img2.jpg"},
|
|
]
|
|
meta = _make_metadata(images=images)
|
|
urls = get_civitai_image_urls(meta)
|
|
self.assertEqual(len(urls), 2)
|
|
self.assertIn("https://civitai.com/img1.jpg", urls)
|
|
|
|
def test_filters_empty_urls(self):
|
|
images = [{"url": ""}, {"url": "https://civitai.com/valid.jpg"}]
|
|
meta = _make_metadata(images=images)
|
|
urls = get_civitai_image_urls(meta)
|
|
self.assertEqual(urls, ["https://civitai.com/valid.jpg"])
|
|
|
|
def test_no_images(self):
|
|
meta = _make_metadata(images=[])
|
|
self.assertEqual(get_civitai_image_urls(meta), [])
|
|
|
|
def test_no_civitai(self):
|
|
self.assertEqual(get_civitai_image_urls({}), [])
|
|
|
|
def test_images_without_url_key(self):
|
|
images = [{"id": 1}, {"url": "https://civitai.com/valid.jpg"}]
|
|
meta = _make_metadata(images=images)
|
|
urls = get_civitai_image_urls(meta)
|
|
self.assertEqual(urls, ["https://civitai.com/valid.jpg"])
|
|
|
|
def test_non_dict_images_skipped(self):
|
|
images = [None, "bad", {"url": "https://civitai.com/valid.jpg"}]
|
|
meta = _make_metadata(images=images)
|
|
urls = get_civitai_image_urls(meta)
|
|
self.assertEqual(urls, ["https://civitai.com/valid.jpg"])
|
|
|
|
|
|
# ── Filesystem-dependent tests ────────────────────────────────────────
|
|
|
|
|
|
class TestReadLoraMetadata(unittest.TestCase):
|
|
"""Test read_lora_metadata — file I/O with JSON parsing."""
|
|
|
|
def test_reads_valid_json(self):
|
|
with tempfile.NamedTemporaryFile(
|
|
mode="w", suffix=".metadata.json", delete=False
|
|
) as f:
|
|
json.dump({"civitai": {"trainedWords": ["test"]}}, f)
|
|
f.flush()
|
|
path = Path(f.name)
|
|
try:
|
|
result = read_lora_metadata(path)
|
|
self.assertIsNotNone(result)
|
|
self.assertEqual(result["civitai"]["trainedWords"], ["test"])
|
|
finally:
|
|
os.unlink(path)
|
|
|
|
def test_returns_none_for_invalid_json(self):
|
|
with tempfile.NamedTemporaryFile(
|
|
mode="w", suffix=".metadata.json", delete=False
|
|
) as f:
|
|
f.write("not valid json {{{")
|
|
f.flush()
|
|
path = Path(f.name)
|
|
try:
|
|
result = read_lora_metadata(path)
|
|
self.assertIsNone(result)
|
|
finally:
|
|
os.unlink(path)
|
|
|
|
def test_returns_none_for_missing_file(self):
|
|
result = read_lora_metadata(Path("/nonexistent/file.metadata.json"))
|
|
self.assertIsNone(result)
|
|
|
|
|
|
class TestGetLoraImageCacheDir(unittest.TestCase):
|
|
"""Test get_lora_image_cache_dir — returns and creates cache path."""
|
|
|
|
def test_returns_path(self):
|
|
cache_dir = get_lora_image_cache_dir()
|
|
self.assertIsInstance(cache_dir, Path)
|
|
self.assertTrue(str(cache_dir).endswith("data/lora_images"))
|
|
|
|
def test_directory_exists(self):
|
|
cache_dir = get_lora_image_cache_dir()
|
|
self.assertTrue(cache_dir.is_dir())
|
|
|
|
|
|
# ── TriggerWordCache tests ────────────────────────────────────────────
|
|
|
|
|
|
class TestTriggerWordCache(unittest.TestCase):
|
|
"""Test TriggerWordCache — thread-safe trigger word lookup."""
|
|
|
|
def setUp(self):
|
|
self.cache = TriggerWordCache()
|
|
|
|
def test_initial_state(self):
|
|
self.assertFalse(self.cache.is_loaded)
|
|
self.assertEqual(self.cache.get_trigger_words("anything"), [])
|
|
|
|
def test_load_from_temp_directory(self):
|
|
"""Create temp metadata files and verify cache loads them."""
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
# Create a metadata file
|
|
meta = {
|
|
"file_name": "my_lora.safetensors",
|
|
"civitai": {"trainedWords": ["trigger1", "trigger2"]},
|
|
}
|
|
meta_path = Path(tmpdir) / "my_lora.safetensors.metadata.json"
|
|
meta_path.write_text(json.dumps(meta))
|
|
|
|
# Patch find_lora_directories to return our temp dir
|
|
with patch("py.lora_utils.find_lora_directories", return_value=[tmpdir]):
|
|
count = self.cache.load(tmpdir)
|
|
|
|
self.assertTrue(self.cache.is_loaded)
|
|
# Cache keys by both file_name stem and metadata filename stem
|
|
self.assertGreaterEqual(count, 1)
|
|
self.assertEqual(
|
|
self.cache.get_trigger_words("my_lora"), ["trigger1", "trigger2"]
|
|
)
|
|
|
|
def test_case_insensitive_lookup(self):
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
meta = {
|
|
"file_name": "MyLoRA.safetensors",
|
|
"civitai": {"trainedWords": ["word1"]},
|
|
}
|
|
(Path(tmpdir) / "MyLoRA.safetensors.metadata.json").write_text(
|
|
json.dumps(meta)
|
|
)
|
|
|
|
with patch("py.lora_utils.find_lora_directories", return_value=[tmpdir]):
|
|
self.cache.load(tmpdir)
|
|
|
|
self.assertEqual(self.cache.get_trigger_words("mylora"), ["word1"])
|
|
self.assertEqual(self.cache.get_trigger_words("MYLORA"), ["word1"])
|
|
|
|
def test_clear(self):
|
|
# Manually set cache state
|
|
self.cache._cache = {"test": ["word"]}
|
|
self.cache._loaded = True
|
|
|
|
self.cache.clear()
|
|
self.assertFalse(self.cache.is_loaded)
|
|
self.assertEqual(self.cache.get_trigger_words("test"), [])
|
|
|
|
def test_unknown_lora_returns_empty(self):
|
|
self.cache._cache = {"known": ["word"]}
|
|
self.cache._loaded = True
|
|
self.assertEqual(self.cache.get_trigger_words("unknown"), [])
|
|
|
|
def test_thread_safety(self):
|
|
"""Verify concurrent access doesn't raise."""
|
|
self.cache._cache = {"lora": ["word"]}
|
|
self.cache._loaded = True
|
|
|
|
errors = []
|
|
|
|
def reader():
|
|
try:
|
|
for _ in range(100):
|
|
self.cache.get_trigger_words("lora")
|
|
except Exception as e:
|
|
errors.append(e)
|
|
|
|
threads = [threading.Thread(target=reader) for _ in range(10)]
|
|
for t in threads:
|
|
t.start()
|
|
for t in threads:
|
|
t.join()
|
|
|
|
self.assertEqual(errors, [])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|