Files
oliverswitzer-ComfyUI-Lora-…/tests/test_lora_metadata_utils.py
T
Oliver Switzer 5db1910e02 feat: add comprehensive PRD for LoRA Prompt Composer node
- Define technical architecture using sentence-transformers embeddings
- Specify content support for creative content creation
- Detail LoRA discovery and prompt composition features
- Include weight optimization and style mimicry requirements

Update .cursorrules

feat: extend LoRA metadata utils with discovery and analysis functions

- Add discover_all_loras() for scanning LoRA directory
- Add extract_embeddable_content() for creating searchable text
- Add extract_example_prompts() for style analysis
- Add classify_lora_type() to differentiate image vs video LoRAs
- Add extract_recommended_weight() with pattern matching
- Comprehensive test coverage for all new functions
- Maintains backward compatibility with existing nodes

feat: implement LoRA Prompt Composer node with semantic search

- Create PromptComposerNode with natural language scene description input
- Implement intelligent LoRA discovery using sentence-transformers embeddings
- Support separate limits for image and video LoRAs
- Add automatic weight optimization based on metadata analysis
- Include trigger word integration and style mimicry from examples
- Register node in ComfyUI with proper display name mapping
- Comprehensive test coverage with mocked dependencies
- Full content support without filtering
- Follows ComfyUI architectural patterns and code standards

docs: add development status summary for LoRA Prompt Composer

- Document completed implementation with all features
- Include technical specifications and usage instructions
- Highlight performance characteristics and architecture
- Confirm readiness for deployment and testing

feat: add required dependencies for LoRA Prompt Composer

- Add sentence-transformers>=2.2.0 for embeddings model
- Add scikit-learn>=1.0.0 for cosine similarity calculations
- Update test expectations to match actual requests module usage
- All 79 tests passing with new dependencies installed

Add cursorrule for using SOLID principles

debug: add detailed logging to LoRA classification and relevance scoring

- Add classification logs to see which LoRAs are found and how they're categorized
- Add relevance scoring logs to debug similarity thresholds
- Add summary logs to understand filtering results
- Fix base_model matching for 'SD 1.5' format in metadata

fix: lower default relevance threshold from 0.7 to 0.3

- 0.7 threshold was too high for semantic similarity scores
- Even highly relevant LoRAs typically score 0.3-0.5 with sentence transformers
- Remove debug logging now that issue is identified and fixed
- Based on testing: pose LoRAs scored appropriate similarity ranges

fix: add defensive programming for None trigger_words and metadata

- Handle case where trigger_words might be None instead of empty list
- Add proper None checking for metadata in extract_example_prompts
- Prevent 'argument of type NoneType is not iterable' runtime error
- Add error logging for invalid trigger_words data

fix: handle None meta field in LoRA image metadata

- Fix 'argument of type NoneType is not iterable' error in extract_example_prompts
- Add explicit None check for image['meta'] before accessing prompt field
- Some LoRAs have images with meta: None (e.g., video LoRAs)
- Add comprehensive debug logging to identify exact error location

feat: prioritize LoRA title/filename words in semantic matching

- Extract meaningful words from file_name and repeat 3x for higher weight
- Extract and include words from model_name for additional context
- Filter out common technical terms (lora, wan, lownoise, i2v, etc.)
- This should improve position-specific matching for pose LoRAs
- Example: pose terms now appear 4x in relevant LoRA embeddings

debug: add comprehensive logging for embedding generation and similarity scoring

- Log file name extraction and word filtering process
- Show which words get 3x priority boost from title
- Display final embeddable content and word counts
- Log base similarity scores and content boost applications
- This will help debug why pose LoRAs aren't ranking higher

fix: expand content boost criteria for pose-specific LoRAs

- Add 'pose', 'position', 'style', 'concept' and related terms to content_tags
- This ensures pose LoRAs get content boost like other specialized LoRAs
- Pose LoRAs now correctly get prioritized for relevant queries
- Refactor long lines for pylint compliance

refactor: remove verbose debug logging in prompt composition

- Remove 'Processing image/video LoRA X: {full_dict}' logs that dumped entire metadata
- Remove individual trigger word processing logs that created noise
- Keep high-level progress logs for debugging
- Significantly reduces log noise during prompt composition

feat: add keyword-specific boosting for position matching

- Implement _get_keyword_boost() method for exact position/concept matches
- 30% boost when LoRA name matches position keywords
- 20% boost when LoRA tags match position keywords
- Increase default content_boost from 1.0 to 1.2 (20% base boost)
- Now pose queries should prioritize relevant LoRAs over generic ones
- Supports various pose and style keyword matching

debug: add detailed logging for keyword boost debugging

- Add step-by-step logs to understand why keyword boost isn't applying
- Show scene text, LoRA name, and position matching checks
- This will help identify why pose LoRAs aren't getting keyword boost

enhance: improve keyword boost scoring and add comprehensive logging

- Increase title match boost to 1.5x (50% boost)
- Increase tag match boost to 1.3x (30% boost)
- Add detailed logging for debugging keyword boost application
- Show step-by-step position matching for LoRA names, tags, and civitai tags

fix: prevent stacking multiple keyword boosts for same position

- Replace multiple boost stacking with single best boost selection
- Title matches get highest priority (1.5x) over tag matches (1.3x)
- Prevents style LoRAs from getting excessive boost stacking
- Now specific LoRAs should correctly rank for targeted queries
- Add detailed logging to show which boost source is selected

fix: prevent duplicate LoRA tags by cleaning scene description

- Strip existing <lora:> and <wanlora:> tags from scene description before processing
- Apply cleaning at 3 key points: embedding generation, keyword boost, and final prompt
- Prevents duplicate LoRAs like '<lora:StyleLora:0.8> + <lora:StyleLora:1>'
- Users can now input prompts with existing LoRA tags without duplication
- Add detailed logging to show original vs cleaned scene descriptions

refactor: remove noisy debug logs from LoRA visualizer

- Remove 'DEBUG: Sending LoRA data to frontend' log that dumps entire data structure
- Remove 'DEBUG: Parsed X standard LoRAs' logs with full LoRA lists
- Keep essential error logging but reduce console noise during normal operation
- Improves log readability for users
2025-08-11 13:18:53 -04:00

319 lines
13 KiB
Python

"""
Unit tests for the LoRA metadata utilities.
These tests verify that the shared metadata loading and parsing functionality
works correctly with mock data and different metadata file formats.
"""
import os
import unittest
from unittest.mock import patch, mock_open
from nodes.lora_metadata_utils import (
LoRAMetadataLoader,
load_lora_metadata,
get_lora_trigger_words,
is_video_lora,
extract_embeddable_content,
extract_example_prompts,
classify_lora_type,
extract_recommended_weight,
)
os.environ.setdefault("COMFYUI_SKIP_LORA_ANALYSIS", "1")
class TestLoRAMetadataLoader(unittest.TestCase):
"""Unit tests for LoRAMetadataLoader class."""
def setUp(self):
self.loader = LoRAMetadataLoader()
def test_extract_trigger_words_from_civitai(self):
"""extract_trigger_words should extract from civitai.trainedWords."""
metadata = {
"civitai": {"trainedWords": ["beautiful", "anime-style", "portrait"]}
}
trigger_words = self.loader.extract_trigger_words(metadata)
self.assertEqual(len(trigger_words), 3)
self.assertIn("beautiful", trigger_words)
self.assertIn("anime-style", trigger_words)
self.assertIn("portrait", trigger_words)
def test_extract_trigger_words_empty_metadata(self):
"""extract_trigger_words should return empty list for None or missing data."""
self.assertEqual(self.loader.extract_trigger_words(None), [])
self.assertEqual(self.loader.extract_trigger_words({}), [])
self.assertEqual(self.loader.extract_trigger_words({"civitai": {}}), [])
def test_extract_trigger_words_filters_empty_strings(self):
"""extract_trigger_words should filter out empty or whitespace-only strings."""
metadata = {
"civitai": {"trainedWords": ["beautiful", "", " ", "anime-style", None]}
}
trigger_words = self.loader.extract_trigger_words(metadata)
self.assertEqual(len(trigger_words), 2)
self.assertIn("beautiful", trigger_words)
self.assertIn("anime-style", trigger_words)
def test_is_video_lora_base_model(self):
"""is_video_lora should detect video LoRAs from base_model field."""
video_metadata = {"base_model": "Wan Video 14B i2v 480p"}
image_metadata = {"base_model": "Stable Diffusion XL"}
self.assertTrue(self.loader.is_video_lora(video_metadata))
self.assertFalse(self.loader.is_video_lora(image_metadata))
def test_is_video_lora_civitai_base_model(self):
"""is_video_lora should detect video LoRAs from civitai.baseModel field."""
video_metadata = {"civitai": {"baseModel": "Wan Video 14B i2v 480p"}}
image_metadata = {"civitai": {"baseModel": "SDXL 1.0"}}
self.assertTrue(self.loader.is_video_lora(video_metadata))
self.assertFalse(self.loader.is_video_lora(image_metadata))
def test_is_video_lora_case_insensitive(self):
"""is_video_lora should be case insensitive."""
metadata = {"base_model": "WAN VIDEO 14B I2V 480P"}
self.assertTrue(self.loader.is_video_lora(metadata))
def test_is_video_lora_empty_metadata(self):
"""is_video_lora should return False for None or empty metadata."""
self.assertFalse(self.loader.is_video_lora(None))
self.assertFalse(self.loader.is_video_lora({}))
@patch("nodes.lora_metadata_utils.folder_paths")
@patch("builtins.open", new_callable=mock_open)
@patch("os.path.exists")
def test_load_metadata_success(self, mock_exists, mock_file, mock_folder_paths):
"""load_metadata should successfully load JSON from metadata file."""
# Mock folder_paths
mock_folder_paths.get_folder_paths.return_value = ["/fake/loras"]
# Mock file existence
mock_exists.return_value = True
# Mock file content
mock_metadata = {
"model_name": "Test LoRA",
"civitai": {"trainedWords": ["test"]},
}
mock_file.return_value.read.return_value = (
'{"model_name": "Test LoRA", "civitai": {"trainedWords": ["test"]}}'
)
# Create new loader to pick up mocked folder_paths
loader = LoRAMetadataLoader()
result = loader.load_metadata("test-lora")
self.assertIsNotNone(result)
self.assertEqual(result["model_name"], "Test LoRA")
@patch("nodes.lora_metadata_utils.folder_paths", None)
def test_load_metadata_no_folder_paths(self):
"""load_metadata should handle missing folder_paths gracefully."""
loader = LoRAMetadataLoader()
result = loader.load_metadata("test-lora")
self.assertIsNone(result)
def test_get_lora_info(self):
"""get_lora_info should combine trigger words and video detection."""
mock_metadata = {
"base_model": "Wan Video 14B i2v",
"civitai": {"trainedWords": ["motion-blur", "cinematic"]},
}
with patch.object(self.loader, "load_metadata", return_value=mock_metadata):
info = self.loader.get_lora_info("test-lora")
self.assertEqual(info["trigger_words"], ["motion-blur", "cinematic"])
self.assertTrue(info["is_video_lora"])
self.assertEqual(info["metadata"], mock_metadata)
class TestConvenienceFunctions(unittest.TestCase):
"""Test the convenience functions that use the global loader."""
@patch("nodes.lora_metadata_utils.get_metadata_loader")
def test_load_lora_metadata(self, mock_get_loader):
"""load_lora_metadata should delegate to the global loader."""
mock_loader = mock_get_loader.return_value
mock_loader.load_metadata.return_value = {"test": "data"}
result = load_lora_metadata("test-lora")
mock_loader.load_metadata.assert_called_once_with("test-lora")
self.assertEqual(result, {"test": "data"})
@patch("nodes.lora_metadata_utils.get_metadata_loader")
def test_get_lora_trigger_words(self, mock_get_loader):
"""get_lora_trigger_words should return trigger words for a LoRA."""
mock_loader = mock_get_loader.return_value
mock_loader.load_metadata.return_value = {"civitai": {"trainedWords": ["test"]}}
mock_loader.extract_trigger_words.return_value = ["test"]
result = get_lora_trigger_words("test-lora")
self.assertEqual(result, ["test"])
@patch("nodes.lora_metadata_utils.get_metadata_loader")
def test_is_video_lora_convenience(self, mock_get_loader):
"""is_video_lora convenience function should work correctly."""
mock_loader = mock_get_loader.return_value
mock_loader.load_metadata.return_value = {"base_model": "Wan Video"}
mock_loader.is_video_lora.return_value = True
result = is_video_lora("test-lora")
self.assertTrue(result)
class TestNewMetadataFunctions(unittest.TestCase):
"""Tests for the new metadata utility functions."""
def test_extract_embeddable_content_basic(self):
"""extract_embeddable_content should combine various metadata fields."""
metadata = {
"model_name": "Test LoRA",
"modelDescription": "<p>A great <strong>style</strong> LoRA</p>",
"tags": ["anime", "portrait"],
"civitai": {
"model": {
"name": "Civitai Model",
"description": "<h1>Detailed</h1> description",
"tags": ["character", "detailed"],
},
"trainedWords": ["trigger1", "trigger2"],
},
}
content = extract_embeddable_content(metadata)
self.assertIn("Test LoRA", content)
self.assertIn("A great style LoRA", content) # HTML stripped
self.assertIn("Civitai Model", content)
self.assertIn("Detailed description", content) # HTML stripped
self.assertIn("anime", content)
self.assertIn("portrait", content)
self.assertIn("character", content)
self.assertIn("detailed", content)
self.assertIn("trigger1", content)
self.assertIn("trigger2", content)
def test_extract_embeddable_content_empty(self):
"""extract_embeddable_content should handle empty metadata gracefully."""
self.assertEqual(extract_embeddable_content({}), "")
self.assertEqual(extract_embeddable_content({"civitai": {}}), "")
def test_extract_example_prompts(self):
"""extract_example_prompts should extract prompts from civitai images."""
metadata = {
"civitai": {
"images": [
{"meta": {"prompt": "beautiful anime girl, detailed"}},
{"meta": {"prompt": "cyberpunk scene, neon"}},
{"meta": {"prompt": ""}}, # Empty prompt should be skipped
{"meta": {}}, # No prompt field
{"meta": {"prompt": "landscape, mountains"}},
]
}
}
prompts = extract_example_prompts(metadata, limit=3)
self.assertEqual(len(prompts), 3)
self.assertIn("beautiful anime girl, detailed", prompts)
self.assertIn("cyberpunk scene, neon", prompts)
self.assertIn("landscape, mountains", prompts)
def test_extract_example_prompts_empty(self):
"""extract_example_prompts should handle empty metadata gracefully."""
self.assertEqual(extract_example_prompts({}), [])
self.assertEqual(extract_example_prompts({"civitai": {}}), [])
self.assertEqual(extract_example_prompts({"civitai": {"images": []}}), [])
def test_classify_lora_type_video(self):
"""classify_lora_type should identify video LoRAs correctly."""
metadata_variants = [
{"base_model": "Wan Video 14B i2v 480p"},
{"base_model": "Video Generation Model"},
{"civitai": {"baseModel": "Wan Video"}},
{"civitai": {"baseModel": "I2V Model"}},
]
for metadata in metadata_variants:
with self.subTest(metadata=metadata):
self.assertEqual(classify_lora_type(metadata), "video")
def test_classify_lora_type_image(self):
"""classify_lora_type should identify image LoRAs correctly."""
metadata_variants = [
{"base_model": "SDXL 1.0"},
{"base_model": "Illustrious"},
{"base_model": "NoobAI"},
{"civitai": {"baseModel": "FLUX.1"}},
{"civitai": {"baseModel": "SD1.5"}},
]
for metadata in metadata_variants:
with self.subTest(metadata=metadata):
self.assertEqual(classify_lora_type(metadata), "image")
def test_classify_lora_type_unknown(self):
"""classify_lora_type should return unknown for unrecognized models."""
self.assertEqual(classify_lora_type({}), "unknown")
self.assertEqual(classify_lora_type({"base_model": "Unknown Model"}), "unknown")
def test_extract_recommended_weight_from_description(self):
"""extract_recommended_weight should parse weights from descriptions."""
test_cases = [
("Best result with weight between : 0.3-0.7", 0.3),
("Recommended strength: 0.8", 0.8),
("Use 0.6 strength for best results", 0.6),
("Weight 1.2 works well", 1.2),
("Best at 0.9", 0.9),
]
for description, expected_weight in test_cases:
with self.subTest(description=description):
metadata = {"modelDescription": description}
weight = extract_recommended_weight(metadata)
self.assertEqual(weight, expected_weight)
def test_extract_recommended_weight_defaults(self):
"""extract_recommended_weight should use appropriate defaults."""
# Video LoRA default
video_metadata = {"base_model": "Wan Video"}
self.assertEqual(extract_recommended_weight(video_metadata), 0.6)
# Image LoRA default
image_metadata = {"base_model": "SDXL"}
self.assertEqual(extract_recommended_weight(image_metadata), 0.8)
# Unknown LoRA default
unknown_metadata = {}
self.assertEqual(extract_recommended_weight(unknown_metadata), 0.8)
def test_extract_recommended_weight_bounds_checking(self):
"""extract_recommended_weight should reject unreasonable weights."""
# Too high
metadata = {"modelDescription": "Use weight 5.0"}
self.assertEqual(
extract_recommended_weight(metadata), 0.8
) # Should use default
# Too low
metadata = {"modelDescription": "Use weight 0.05"}
self.assertEqual(
extract_recommended_weight(metadata), 0.8
) # Should use default
if __name__ == "__main__":
unittest.main()