diff --git a/RUN_TESTS.md b/RUN_TESTS.md index 28de3ca..8e26afa 100644 --- a/RUN_TESTS.md +++ b/RUN_TESTS.md @@ -1,122 +1,147 @@ # Running Tests -Quick guide to running tests for the LoRA-Merger-ComfyUI custom nodes. +The test suite runs **without a ComfyUI installation**. ComfyUI's modules +(`comfy`, `folder_paths`, `comfy_extras`, `nodes`) are mocked in +[tests/conftest.py](tests/conftest.py), so the only requirements are torch, +mergekit and pytest. -## Integration Test - -### Quick Start +## Quick Start ```bash cd /home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py +/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m pytest tests/ ``` -### What It Tests +Expected: **238 passed**. -The integration test validates the **PM LoRA Semantic Analyzer (Gradient)** pipeline: +Any Python with the dependencies below works; `/home/lars/SD/Apps/ComfyUI/.venv` +is simply the environment that already has them on this machine. -1. ✅ Dependencies (PyTorch, diffusers, safetensors) -2. ✅ Node instantiation (all semantic merge nodes) -3. ✅ INPUT_TYPES configuration -4. ✅ Checkpoint dropdown population -5. ✅ Specification parsing - -### Expected Output - -``` -================================================================================ -PM LoRA Semantic Analyzer (Gradient) - Integration Test -================================================================================ - -[... file checks ...] - -================================================================================ -TEST SUMMARY -================================================================================ -✓ PASS Dependencies -✓ PASS LoRA Power Stacker -✓ PASS Semantic Analyzer Instantiation -✓ PASS Semantic Merge Spec -✓ PASS Semantic Merger Instantiation - -Results: 5/5 tests passed - -🎉 ALL TESTS PASSED! -``` - -### Exit Codes - -- `0` - All tests passed -- `1` - One or more tests failed -- `130` - Test interrupted (Ctrl+C) - -## Unit Tests - -### Run All Unit Tests +### Run a single file or test ```bash -cd /home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/ +PY=/home/lars/SD/Apps/ComfyUI/.venv/bin/python + +$PY -m pytest tests/test_validation.py # one file +$PY -m pytest tests/test_gta_sparsify.py -v # verbose +$PY -m pytest tests/test_types.py::TestTypeGuards # one class +$PY -m pytest tests/ -k "sparsify or interp" # by name ``` -### Run Specific Test Files +### Coverage ```bash -# Test types system -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_types.py -v - -# Test decomposition -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_decomposition.py -v - -# Test validation -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_validation.py -v - -# Test algorithms -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_algorithms.py -v +/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m pytest --cov=src tests/ ``` -### Run with Coverage +Requires `pytest-cov` (see `requirements-dev.txt`). + +## Two Kinds of Test File + +Both kinds are collected by `pytest tests/`; the difference only matters when +you want to run one directly. + +**Script-style** — plain asserts driven by a `run([...])` list, guarded behind +`if __name__ == "__main__":`. These also run as standalone scripts with no +pytest at all, which is useful for a quick check or when bisecting: ```bash -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest --cov=src tests/ +/home/lars/SD/Apps/ComfyUI/.venv/bin/python tests/test_gta_sparsify.py +# -> PASS magnitude ... All 12 passed ``` -## Syntax Validation +Files: `test_gta_behavior`, `test_gta_merge`, `test_gta_parity`, +`test_gta_sparsify`, `test_interp_delta_merge`, `test_interp_fidelity`, +`test_interp_integration`, `test_lora_save`, `test_merge_node_names`, +`test_merger_vram_offload`. -### Check Python Syntax +**Pytest-style** — `Test*` classes, fixtures, `pytest.raises`. Run these through +pytest only; executing them directly does not set up the import paths. -```bash -# Single file -python3 -m py_compile src/nodes_semantic_merge.py +Files: `test_algorithms`, `test_blocks`, `test_decomposition`, +`test_spectral_norm`, `test_types`, `test_utility`, `test_validation`. -# All Python files -find . -name "*.py" -not -path "./venv/*" -exec python3 -m py_compile {} \; -``` +## How Imports Are Set Up -### Check with Flake8 +Two rules keep the suite importable outside ComfyUI. Both are handled by +[tests/conftest.py](tests/conftest.py); they matter when you add a test. -```bash -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m flake8 src/ --max-line-length=120 -``` +1. **Import project code as `src.`**, never as a bare top-level module: + + ```python + from src.validation import LoRAStackValidator # correct + from validation import LoRAStackValidator # breaks + from types import is_lora_tensors # breaks: stdlib `types` + ``` + + Modules under `src/` use package-relative imports (`from ..types import ...`), + which only resolve when `src` itself is the package. conftest puts the project + root on `sys.path` for this. The same applies to `unittest.mock.patch` + targets: `@patch('src.merge.algorithms.LinearMergeTask')`. + +2. **Do not add `tests/__init__.py`.** With it, pytest walks up to the project + root's `__init__.py` — the ComfyUI node entry point — and imports ComfyUI + before any test runs. + +`conftest.py` lives in `tests/`, not the project root, for the same reason: a +root-level conftest is imported as part of that package. ## Test Files -### Integration Tests -- `test_gradient_analyzer_integration.py` - Validates semantic analyzer pipeline +| File | Covers | +|------|--------| +| `test_algorithms.py` | Merge algorithm registry and dispatch | +| `test_blocks.py` | Block selection, key normalization, block weights | +| `test_decomposition.py` | SVD / randomized SVD / energy-based decomposers | +| `test_gta_behavior.py` | GTA delta-space merge behavior | +| `test_gta_merge.py` | Sign election and disjoint merge | +| `test_gta_parity.py` | Parity of local sparsify vs. mergekit | +| `test_gta_sparsify.py` | magnitude / outliers / bernoulli / della sparsify | +| `test_interp_delta_merge.py` | slerp / nuslerp / karcher / nearswap in delta space | +| `test_interp_fidelity.py` | Delta-space blend vs. reference average | +| `test_interp_integration.py` | End-to-end merge through the node | +| `test_lora_save.py` | Tensor sanitization before safetensors save | +| `test_merge_node_names.py` | Node widget names and titles | +| `test_merger_vram_offload.py` | `offload_models` widget behavior | +| `test_spectral_norm.py` | Spectral norm scaling | +| `test_types.py` | Type guards and validators | +| `test_utility.py` | SVD/QR pipeline helpers | +| `test_validation.py` | LoRA stack, shape and parameter validation | -### Unit Tests (in `tests/` directory) -- `test_types.py` - Type system and validators -- `test_algorithms.py` - Merge algorithms -- `test_decomposition.py` - SVD/QR decomposition -- `test_validation.py` - Input validation +## Requirements -### Test Workflows -- `/home/lars/SD/Apps/ComfyUI/user/default/workflows/PM-Gradient-Analyzer-zImage.json` - Example workflow +```bash +pip install -r requirements-dev.txt +``` + +Needed to run the suite: `torch`, `mergekit`, `pytest`. Optional: +`pytest-cov` (coverage), `pytest-xdist` (`-n auto` parallel runs). + +A ComfyUI installation is **not** required. + +## Syntax and Lint + +```bash +find src -name "*.py" -exec python3 -m py_compile {} \; +/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m flake8 src/ --max-line-length=120 +``` + +## Debugging Failures + +```bash +PY=/home/lars/SD/Apps/ComfyUI/.venv/bin/python + +$PY -m pytest tests/ -vv --tb=long # full tracebacks +$PY -m pytest tests/ -x # stop at first failure +$PY -m pytest tests/ --lf # rerun last failures +``` + +If a new test fails at import with `attempted relative import beyond top-level +package`, `No module named 'src'`, or picks up the stdlib `types` module, it is +importing project code as a top-level module — see "How Imports Are Set Up". ## Continuous Integration -### GitHub Actions Example - ```yaml name: Tests @@ -126,138 +151,17 @@ jobs: test: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v4 - - name: Set up Python - uses: actions/setup-python@v2 + - uses: actions/setup-python@v5 with: - python-version: '3.10' + python-version: '3.11' - name: Install dependencies - run: | - pip install -r requirements.txt - pip install pytest pytest-cov + run: pip install -r requirements-dev.txt - - name: Run integration tests - run: python3 test_gradient_analyzer_integration.py - - - name: Run unit tests - run: pytest tests/ -v --cov=src + - name: Run tests + run: pytest tests/ -v ``` -## Test Requirements - -### For Integration Tests - -**Required:** -- ComfyUI installed at `/home/lars/SD/Apps/ComfyUI` -- venv at `/home/lars/SD/Apps/ComfyUI/venv` -- Dependencies: `torch`, `diffusers>=0.30.0`, `safetensors` - -**Optional (for file checks):** -- zImage checkpoint in `models/diffusion_models/` -- LoRAs in `models/loras/` - -### For Unit Tests - -**Required:** -- PyTorch -- Standard Python libraries - -**Optional:** -- pytest -- pytest-cov (for coverage reports) - -## Quick Commands Reference - -```bash -# Integration test (basic validation) -./test_gradient_analyzer_integration.py - -# Unit tests (comprehensive) -pytest tests/ -v - -# Syntax check -python3 -m py_compile src/*.py - -# Coverage report -pytest --cov=src --cov-report=html tests/ - -# Specific test -pytest tests/test_types.py::test_validate_lora_tensors -v -``` - -## Debugging Test Failures - -### Enable Verbose Output - -```bash -# Integration test -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py 2>&1 | tee test_output.log - -# Unit tests -pytest tests/ -vv --tb=long -``` - -### Check Imports - -```bash -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c " -import sys -sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI') -sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI') -from src.nodes_semantic_merge import PMLoRASemanticAnalyzerGradient -print('✓ Import successful') -" -``` - -### Test Individual Components - -```bash -# Test diffusers -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c " -from diffusers.models import ZImageTransformer2DModel -print('✓ ZImageTransformer2DModel available') -" - -# Test folder_paths -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c " -import sys -sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI') -import folder_paths -print('✓ folder_paths available') -" -``` - -## Performance Testing - -### Time Individual Tests - -```bash -time /home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py -``` - -### Profile Tests - -```bash -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m cProfile -o test_profile.prof test_gradient_analyzer_integration.py - -# View profile -/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pstats test_profile.prof -``` - -## Documentation - -- **TEST_GRADIENT_ANALYZER.md** - Detailed test documentation -- **QUICK_START_GRADIENT_ANALYZER.md** - User guide -- **ZIMAGE_LOADER_OPTIMIZATION.md** - Technical details -- **BUGFIX_ZIMAGE_IMPORT.md** - Import fix documentation - -## Support - -For issues or questions: -1. Check test output for error messages -2. Review documentation files -3. Verify all dependencies are installed -4. Check that files exist in expected locations -5. Open an issue on GitHub if problems persist +No ComfyUI checkout is needed in CI. diff --git a/run_pytest.py b/run_pytest.py deleted file mode 100755 index 9fd48fa..0000000 --- a/run_pytest.py +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env python3 -""" -Test runner that sets up mocking before pytest runs -""" - -import sys -from unittest.mock import MagicMock -from typing import Tuple -import torch - -# Mock ComfyUI modules BEFORE any imports -sys.modules['comfy'] = MagicMock() -sys.modules['comfy.utils'] = MagicMock() -sys.modules['comfy.model_management'] = MagicMock() -sys.modules['comfy.lora'] = MagicMock() - -# Mock architectures module -architectures_mock = MagicMock() -architectures_mock.sd_lora.UP_DOWN_ALPHA_TUPLE = Tuple[torch.Tensor, torch.Tensor, float] -sys.modules['architectures'] = architectures_mock -sys.modules['architectures.sd_lora'] = architectures_mock.sd_lora -sys.modules['architectures.general_architecture'] = MagicMock() -sys.modules['architectures.wan_lora'] = MagicMock() - -# Now run pytest -import pytest - -if __name__ == "__main__": - sys.exit(pytest.main(sys.argv[1:] or ["test_utility.py", "-v"])) \ No newline at end of file diff --git a/src/decomposition/__init__.py b/src/decomposition/__init__.py index c5c2985..7b99650 100644 --- a/src/decomposition/__init__.py +++ b/src/decomposition/__init__.py @@ -16,7 +16,6 @@ from .svd import ( SVDDecomposer, RandomizedSVDDecomposer, EnergyBasedRandomizedSVDDecomposer, - QRDecomposer, ) __all__ = [ @@ -28,5 +27,4 @@ __all__ = [ 'SVDDecomposer', 'RandomizedSVDDecomposer', 'EnergyBasedRandomizedSVDDecomposer', - 'QRDecomposer', ] diff --git a/src/validation/validators.py b/src/validation/validators.py index c5804d4..ca76b8a 100644 --- a/src/validation/validators.py +++ b/src/validation/validators.py @@ -138,6 +138,24 @@ class TensorShapeValidator: # Check all tensors against reference for lora_name, (up, down, alpha) in tensors_dict.items(): + # Validate up/down dimensions are compatible. This is a per-LoRA + # (self-consistency) check, so it runs for the reference tensor too -- + # otherwise a single incompatible LoRA passes validation unexamined. + if len(up.shape) >= 2 and len(down.shape) >= 2: + up_rank = up.shape[1] + down_rank = down.shape[0] + + if up_rank != down_rank: + errors.append({ + "code": "INCOMPATIBLE_DIMENSIONS", + "message": ( + f"LoRA '{lora_name}' has incompatible up/down dimensions: " + f"up rank {up_rank} != down rank {down_rank}" + ), + "location": lora_name + }) + + # The remaining checks compare against the reference tensor. if lora_name == first_name: continue @@ -165,21 +183,6 @@ class TensorShapeValidator: f"differs from reference {ref_down_shape}" ) - # Validate up/down dimensions are compatible - if len(up.shape) >= 2 and len(down.shape) >= 2: - up_rank = up.shape[1] if len(up.shape) == 2 else up.shape[1] - down_rank = down.shape[0] - - if up_rank != down_rank: - errors.append({ - "code": "INCOMPATIBLE_DIMENSIONS", - "message": ( - f"LoRA '{lora_name}' has incompatible up/down dimensions: " - f"up rank {up_rank} != down rank {down_rank}" - ), - "location": lora_name - }) - return { "valid": len(errors) == 0, "errors": errors, diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index a3cd176..0000000 --- a/tests/__init__.py +++ /dev/null @@ -1,6 +0,0 @@ -""" -Test suite for LoRA Power-Merger. - -Unit tests for core functionality. Integration tests are not included -as they require a running ComfyUI instance. -""" diff --git a/tests/conftest.py b/tests/conftest.py index 4a3c391..59c85ab 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,80 +1,109 @@ -""" -Pytest configuration and fixtures for LoRA Power-Merger tests +"""Pytest configuration for LoRA Power-Merger. -This file sets up the test environment by mocking ComfyUI dependencies -that may not be available in the test environment. +Sets up the test environment so the whole suite runs on torch, mergekit and +pytest alone -- no ComfyUI installation required. + +Deliberately lives in tests/ rather than at the project root: the root holds an +``__init__.py`` (the ComfyUI node entry point, which imports ComfyUI), so pytest +would import a root-level conftest as a submodule of that package and execute the +entry point before any test ran. tests/ is not a package, so this file imports on +its own. """ -import sys import os +import sys +import types from unittest.mock import MagicMock -from typing import Tuple -import torch -# Add the src directory to the path to allow imports -project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) -src_path = os.path.join(project_root, 'src') -if src_path not in sys.path: - sys.path.insert(0, src_path) +# Put the PROJECT ROOT (not src/) on the path, so modules under test are imported +# as `src.`. Modules in src/ use package-relative imports (`from ..types +# import ...`); importing them as top-level modules makes those resolve beyond +# the top-level package and fail. src/__init__.py is empty, so this pulls in no +# ComfyUI dependencies by itself. +PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +if PROJECT_ROOT not in sys.path: + sys.path.insert(0, PROJECT_ROOT) + + +def _stub_root_package(): + """Stop pytest from executing the project root's ``__init__.py``. + + Because the root holds an ``__init__.py``, pytest treats the project root as a + package and imports it in ``Package.setup()`` before running any test under + it. That file is the ComfyUI node entry point (``from .src.lora_apply import + ...``), so importing it outside a ComfyUI process fails on the relative import + -- and it is not test code in any case. + + ``Package.setup()`` goes through ``import_path``, which under the ``importlib`` + import mode returns an already-present ``sys.modules`` entry before touching + the file. The module name it derives for ``/__init__.py`` is + ``"__init__"``; seeding that name with an empty module makes the import a + no-op. Registered here rather than via ``pytest_collect_directory``: that hook + fires only while recursing into directories, not for the parent packages of a + test path named directly on the command line. + """ + sys.modules.setdefault("__init__", types.ModuleType("__init__")) -# Prevent pytest from treating the project root as a package -# by ensuring tests are collected from tests directory only -import pytest def pytest_ignore_collect(collection_path, config): - """Ignore collection of __init__.py files and src/ directory.""" + """Never collect package __init__.py files or anything under src/.""" path_str = str(collection_path) if path_str.endswith('__init__.py'): return True - if '/src/' in path_str or path_str.endswith('/src'): + if (os.sep + 'src' + os.sep) in path_str or path_str.endswith(os.sep + 'src'): return True return False -# Mock ComfyUI modules before any test imports -def pytest_configure(config): - """Configure pytest and mock unavailable modules""" - # Mock comfy module +def _mock_comfyui_modules(): + """Install stand-ins for the ComfyUI modules src/ imports at module scope.""" comfy_mock = MagicMock() sys.modules['comfy'] = comfy_mock - sys.modules['comfy.utils'] = MagicMock() - sys.modules['comfy.model_management'] = MagicMock() - sys.modules['comfy.lora'] = MagicMock() - sys.modules['comfy.weight_adapter'] = MagicMock() - sys.modules['comfy.model_patcher'] = MagicMock() - sys.modules['comfy.sd'] = MagicMock() + for name in ('utils', 'model_management', 'lora', 'weight_adapter', + 'model_patcher', 'sd'): + sys.modules[f'comfy.{name}'] = MagicMock() - # Create a LoRAAdapter mock class class LoRAAdapterMock: - def __init__(self, *args, **kwargs): - self.state_dict = {} + """Stand-in for comfy.weight_adapter.LoRAAdapter. + + Mirrors the real signature: it is a plain container over ``loaded_keys`` + and a ``weights`` 6-tuple (up, down, alpha, mid, dora_scale, reshape), + which the merger reads back after building its output state dict. + """ + + name = "lora" + + def __init__(self, loaded_keys=None, weights=None): + self.loaded_keys = set() if loaded_keys is None else loaded_keys + self.weights = weights comfy_mock.weight_adapter.LoRAAdapter = LoRAAdapterMock + sys.modules['comfy.weight_adapter'].LoRAAdapter = LoRAAdapterMock - # Mock comfy_util module - comfy_util_mock = MagicMock() - sys.modules['comfy_util'] = comfy_util_mock + # The merger logs free VRAM around the offload step, formatting the result + # with `:.0f` -- a bare MagicMock would raise on __format__. + mm = sys.modules['comfy.model_management'] + mm.get_free_memory.return_value = 8 * 1024 ** 3 + comfy_mock.model_management = mm + comfy_mock.utils = sys.modules['comfy.utils'] - # Mock nodes module (ComfyUI nodes) - nodes_mock = MagicMock() - sys.modules['nodes'] = nodes_mock + sys.modules['comfy_extras'] = MagicMock() + sys.modules['comfy_extras.nodes_custom_sampler'] = MagicMock() + sys.modules['nodes'] = MagicMock() - # Mock folder_paths module (ComfyUI utility) folder_paths_mock = MagicMock() folder_paths_mock.get_folder_paths.return_value = [] folder_paths_mock.folder_names_and_paths = {} sys.modules['folder_paths'] = folder_paths_mock - # Mock comfy_extras module (ComfyUI extras) - comfy_extras_mock = MagicMock() - sys.modules['comfy_extras'] = comfy_extras_mock - sys.modules['comfy_extras.nodes_custom_sampler'] = MagicMock() - # Mock architectures module to avoid relative import issues - architectures_mock = MagicMock() - # Define the UP_DOWN_ALPHA_TUPLE type alias - architectures_mock.sd_lora.UP_DOWN_ALPHA_TUPLE = Tuple[torch.Tensor, torch.Tensor, float] - sys.modules['architectures'] = architectures_mock - sys.modules['architectures.sd_lora'] = architectures_mock.sd_lora - sys.modules['architectures.general_architecture'] = MagicMock() - sys.modules['architectures.wan_lora'] = MagicMock() \ No newline at end of file +def pytest_configure(config): + """Mock ComfyUI before test modules (and the src/ modules they import) load.""" + _mock_comfyui_modules() + _stub_root_package() + + +# Also applied at import time: conftest is imported before collection, so this +# covers anything that resolves earlier than pytest_configure. +_mock_comfyui_modules() +_stub_root_package() diff --git a/tests/test_algorithms.py b/tests/test_algorithms.py index 7e0ff85..98f4d68 100644 --- a/tests/test_algorithms.py +++ b/tests/test_algorithms.py @@ -7,19 +7,16 @@ Mock mergekit dependencies to test algorithm logic in isolation. import pytest import torch -import sys -from pathlib import Path from unittest.mock import Mock, MagicMock, patch -# Add src to path for imports -sys.path.insert(0, str(Path(__file__).parent.parent / "src")) - -from merge.algorithms import ( +# Imported as `src.*` so the package-relative imports inside src/ resolve. +# conftest.py puts the project root on sys.path and mocks the ComfyUI modules. +from src.merge.algorithms import ( linear_merge, get_merge_algorithm, MERGE_ALGORITHMS, ) -from merge.utils import apply_weights_to_tensors +from src.merge.utils import apply_weights_to_tensors class TestApplyWeightsToTensors: @@ -116,7 +113,7 @@ class TestAlgorithmRegistry: class TestLinearMerge: """Tests for linear merge algorithm.""" - @patch('merge.algorithms.LinearMergeTask') + @patch('src.merge.algorithms.LinearMergeTask') def test_linear_merge_calls_task(self, mock_task_class): """Test that linear merge creates and executes LinearMergeTask.""" # Setup mocks @@ -153,7 +150,7 @@ class TestLinearMerge: # Verify result assert torch.allclose(result, torch.ones(10, 10)) - @patch('merge.algorithms.LinearMergeTask') + @patch('src.merge.algorithms.LinearMergeTask') def test_linear_merge_default_normalize(self, mock_task_class): """Test that normalize defaults to False if not in method_args.""" mock_task = Mock() diff --git a/tests/test_blocks.py b/tests/test_blocks.py index e7c2a21..1d705bb 100644 --- a/tests/test_blocks.py +++ b/tests/test_blocks.py @@ -1,4 +1,4 @@ -from blocks import normalize_key, parse_weight_list +from src.blocks import normalize_key, parse_weight_list class TestNormalizeKey: @@ -31,7 +31,7 @@ class TestParseWeightList: assert parse_weight_list(None) == [] -from blocks import make_category, key_weight +from src.blocks import make_category, key_weight KREA2_DEF = { "model": "KREA2", @@ -85,7 +85,7 @@ class TestKeyWeight: from collections import OrderedDict -from blocks import compute_lora_weights, merge_selection, build_block_selection_dict, resolve_block_selection +from src.blocks import compute_lora_weights, merge_selection, build_block_selection_dict, resolve_block_selection class TestComputeLoraWeights: @@ -186,7 +186,7 @@ class TestResolveBlockSelection: import torch -from blocks import apply_block_weights +from src.blocks import apply_block_weights class TestApplyBlockWeights: @@ -222,7 +222,7 @@ class TestApplyBlockWeights: assert torch.allclose(up, torch.ones(4, 2)) -from blocks import build_krea2_definition, build_klein_definition +from src.blocks import build_krea2_definition, build_klein_definition class TestBuildKrea2: diff --git a/tests/test_decomposition.py b/tests/test_decomposition.py index 439de4a..c43f43f 100644 --- a/tests/test_decomposition.py +++ b/tests/test_decomposition.py @@ -6,17 +6,13 @@ Tests tensor decomposition functionality including SVD, QR, and error handling. import pytest import torch -import sys -from pathlib import Path -# Add src to path for imports -sys.path.insert(0, str(Path(__file__).parent.parent / "src")) - -from decomposition import ( +# Imported as `src.*` so the package-relative imports inside src/ resolve. +# conftest.py puts the project root on sys.path and mocks the ComfyUI modules. +from src.decomposition import ( SVDDecomposer, RandomizedSVDDecomposer, EnergyBasedRandomizedSVDDecomposer, - QRDecomposer, SingularValueDistribution, ) @@ -26,23 +22,39 @@ class TestSVDDecomposer: def test_basic_2d_decomposition(self): """Test basic 2D tensor decomposition.""" - decomposer = SVDDecomposer() + # return_statistics is a constructor option, not a decompose() argument. + decomposer = SVDDecomposer(return_statistics=True) weight = torch.randn(100, 50) target_rank = 10 - up, down, alpha, stats = decomposer.decompose( - weight, target_rank, return_statistics=True - ) + up, down, alpha, stats = decomposer.decompose(weight, target_rank) # Check shapes assert up.shape == (100, 10) assert down.shape == (10, 50) assert isinstance(alpha, float) - # Check reconstruction quality + # Reconstruction quality is bounded by the discarded singular values, not + # by the decomposer: for a full-rank random Gaussian, a rank-10 truncation + # of 50 singular values necessarily loses ~75% of the Frobenius norm. + # Assert we land at that optimum rather than at an arbitrary threshold. reconstructed = up @ down reconstruction_error = torch.norm(weight - reconstructed) / torch.norm(weight) - assert reconstruction_error < 0.5 # Should be reasonable approximation + + S = torch.linalg.svdvals(weight) + optimal_error = (S[target_rank:].pow(2).sum() / S.pow(2).sum()).sqrt() + assert reconstruction_error < optimal_error + 0.01 + + def test_low_rank_input_reconstructs_accurately(self): + """A genuinely low-rank matrix is recovered with little error.""" + decomposer = SVDDecomposer() + weight = torch.randn(100, 10) @ torch.randn(10, 50) + weight = weight + 0.01 * torch.randn(100, 50) + + up, down, alpha, _ = decomposer.decompose(weight, target_rank=10) + + reconstruction_error = torch.norm(weight - up @ down) / torch.norm(weight) + assert reconstruction_error < 0.05 def test_4d_conv_decomposition(self): """Test 4D convolutional tensor decomposition.""" @@ -77,7 +89,11 @@ class TestSVDDecomposer: recon_sym = up_sym @ down_sym recon_asym = up_asym @ down_asym - assert torch.allclose(recon_sym, recon_asym, rtol=1e-4) + # The product is mathematically identical either way -- the distributions + # only decide whether S goes into up, into down, or is split as sqrt(S) + # across both. atol is needed because the default (1e-8) is below float32 + # rounding for entries near zero; observed difference is ~5e-7. + assert torch.allclose(recon_sym, recon_asym, rtol=1e-4, atol=1e-5) def test_statistics_calculation(self): """Test that statistics are calculated correctly.""" @@ -211,27 +227,6 @@ class TestEnergyBasedRandomizedSVDDecomposer: assert stats_high is not None -class TestQRDecomposer: - """Tests for QR decomposer.""" - - def test_qr_decomposition(self): - """Test basic QR decomposition.""" - decomposer = QRDecomposer() - weight = torch.randn(100, 50) - target_rank = 20 - - up, down, alpha, _ = decomposer.decompose(weight, target_rank) - - # Check shapes - assert up.shape == (100, 20) - assert down.shape == (20, 50) - - # QR decomposition should still provide reasonable approximation - reconstructed = up @ down - reconstruction_error = torch.norm(weight - reconstructed) / torch.norm(weight) - assert reconstruction_error < 1.0 # Looser bound than SVD - - class TestErrorHandling: """Tests for error handling in decomposition.""" @@ -248,11 +243,24 @@ class TestErrorHandling: decomposer = SVDDecomposer(return_statistics=True) weight = torch.zeros(50, 30) - # Should not crash, should handle gracefully + # The zero-matrix guard lives in dynamic rank selection; without a + # dynamic_method the caller has pinned the rank and target_rank is honoured. + up, down, alpha, stats = decomposer.decompose( + weight, target_rank=10, dynamic_method="sv_ratio", dynamic_param=2.0 + ) + + # Rank should collapse to the minimum for a numerically zero matrix + assert stats['new_rank'] == 1 + + def test_zero_matrix_fixed_rank_is_honoured(self): + """A pinned rank (no dynamic_method) is kept even for a zero matrix.""" + decomposer = SVDDecomposer(return_statistics=True) + weight = torch.zeros(50, 30) + up, down, alpha, stats = decomposer.decompose(weight, target_rank=10) - # Rank should be minimal for zero matrix - assert stats['new_rank'] == 1 + assert stats['new_rank'] == 10 + assert torch.allclose(up @ down, torch.zeros(50, 30)) def test_invalid_dynamic_method(self): """Test that invalid dynamic method raises error.""" @@ -290,7 +298,6 @@ class TestIntegrationWithFixtures: SVDDecomposer(), RandomizedSVDDecomposer(), EnergyBasedRandomizedSVDDecomposer(), - QRDecomposer(), ] for decomposer in decomposers: @@ -309,7 +316,6 @@ class TestIntegrationWithFixtures: decomposers = [ SVDDecomposer(), RandomizedSVDDecomposer(), - QRDecomposer(), ] for decomposer in decomposers: diff --git a/tests/test_gta_behavior.py b/tests/test_gta_behavior.py index 2084787..bf080ff 100644 --- a/tests/test_gta_behavior.py +++ b/tests/test_gta_behavior.py @@ -57,8 +57,12 @@ def test_normalize_does_not_collapse_as_1_over_n_squared(): assert merged.norm() > 0.5 * avg.norm() -run([ - ("refactor_reconstructs", test_refactor_reconstructs_delta), - ("nonoverlap_keeps_strength", test_style_plus_character_nonoverlap_keeps_strength), - ("no_1_over_n_squared", test_normalize_does_not_collapse_as_1_over_n_squared), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("refactor_reconstructs", test_refactor_reconstructs_delta), + ("nonoverlap_keeps_strength", test_style_plus_character_nonoverlap_keeps_strength), + ("no_1_over_n_squared", test_normalize_does_not_collapse_as_1_over_n_squared), + ]) diff --git a/tests/test_gta_merge.py b/tests/test_gta_merge.py index 0d868f5..3e7f8b3 100644 --- a/tests/test_gta_merge.py +++ b/tests/test_gta_merge.py @@ -48,10 +48,14 @@ def test_n_equal_loras_scale_one_over_n_not_squared(): assert torch.allclose(merged, d) -run([ - ("elect_sign", test_elect_sign_weighted_majority), - ("nonoverlap_keeps_strength", test_disjoint_normalize_nonoverlap_keeps_strength), - ("conflict_arbitrated", test_disjoint_normalize_overlap_conflict_arbitrated), - ("linear_weighted_average", test_linear_normalize_is_weighted_average), - ("n_loras_1_over_n", test_n_equal_loras_scale_one_over_n_not_squared), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("elect_sign", test_elect_sign_weighted_majority), + ("nonoverlap_keeps_strength", test_disjoint_normalize_nonoverlap_keeps_strength), + ("conflict_arbitrated", test_disjoint_normalize_overlap_conflict_arbitrated), + ("linear_weighted_average", test_linear_normalize_is_weighted_average), + ("n_loras_1_over_n", test_n_equal_loras_scale_one_over_n_not_squared), + ]) diff --git a/tests/test_gta_parity.py b/tests/test_gta_parity.py index 273725d..dfea582 100644 --- a/tests/test_gta_parity.py +++ b/tests/test_gta_parity.py @@ -63,8 +63,12 @@ def test_resolve_rescale_default_matches_mergekit(): assert gta.resolve_rescale_norm("ties", "l2") == "l2" -run([ - ("task_arithmetic_parity", test_task_arithmetic_parity), - ("ties_parity", test_ties_parity), - ("resolve_rescale_default_matches_mergekit", test_resolve_rescale_default_matches_mergekit), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("task_arithmetic_parity", test_task_arithmetic_parity), + ("ties_parity", test_ties_parity), + ("resolve_rescale_default_matches_mergekit", test_resolve_rescale_default_matches_mergekit), + ]) diff --git a/tests/test_gta_sparsify.py b/tests/test_gta_sparsify.py index 7536fd1..a784de7 100644 --- a/tests/test_gta_sparsify.py +++ b/tests/test_gta_sparsify.py @@ -138,17 +138,21 @@ def test_della_wide_layer_preserves_density_and_monotonic(): assert kept_mask[hi].float().mean().item() > kept_mask[~hi].float().mean().item() + 0.05 -run([ - ("magnitude", test_magnitude_matches_mergekit), - ("magnitude_rescale_l1", test_magnitude_rescale_l1_matches), - ("outliers", test_outliers_matches_mergekit), - ("bernoulli_seed", test_bernoulli_matches_with_seed), - ("della_seed", test_della_matches_with_seed), - ("density_one_identity", test_density_one_is_identity), - ("della_large_density", test_della_large_preserves_density), - ("della_large_l1", test_della_large_l1_preserved), - ("della_large_monotonic", test_della_large_keep_prob_monotonic_in_magnitude), - ("della_chunk_density", test_della_chunk_boundary_invariant_density), - ("della_scatter_rank_equivalence", test_della_scatter_rank_equivalence), - ("della_wide_density_monotonic", test_della_wide_layer_preserves_density_and_monotonic), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("magnitude", test_magnitude_matches_mergekit), + ("magnitude_rescale_l1", test_magnitude_rescale_l1_matches), + ("outliers", test_outliers_matches_mergekit), + ("bernoulli_seed", test_bernoulli_matches_with_seed), + ("della_seed", test_della_matches_with_seed), + ("density_one_identity", test_density_one_is_identity), + ("della_large_density", test_della_large_preserves_density), + ("della_large_l1", test_della_large_l1_preserved), + ("della_large_monotonic", test_della_large_keep_prob_monotonic_in_magnitude), + ("della_chunk_density", test_della_chunk_boundary_invariant_density), + ("della_scatter_rank_equivalence", test_della_scatter_rank_equivalence), + ("della_wide_density_monotonic", test_della_wide_layer_preserves_density_and_monotonic), + ]) diff --git a/tests/test_interp_delta_merge.py b/tests/test_interp_delta_merge.py index 41a0528..11e7c7c 100644 --- a/tests/test_interp_delta_merge.py +++ b/tests/test_interp_delta_merge.py @@ -118,10 +118,14 @@ def run(tests): print(f"\nAll {len(tests)} passed") -run([ - ("modes_constant", test_modes_constant), - ("additive_is_blend_times_sum", test_additive_is_blend_times_sum), - ("average_is_blend_times_mean", test_average_is_blend_times_mean), - ("strength_linear_gain_additive", test_strength_is_linear_gain_additive), - ("single_owner_fallback", test_single_owner_fallback_not_zero), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("modes_constant", test_modes_constant), + ("additive_is_blend_times_sum", test_additive_is_blend_times_sum), + ("average_is_blend_times_mean", test_average_is_blend_times_mean), + ("strength_linear_gain_additive", test_strength_is_linear_gain_additive), + ("single_owner_fallback", test_single_owner_fallback_not_zero), + ]) diff --git a/tests/test_interp_fidelity.py b/tests/test_interp_fidelity.py index b800fe8..a3a98e1 100644 --- a/tests/test_interp_fidelity.py +++ b/tests/test_interp_fidelity.py @@ -61,4 +61,8 @@ def run(tests): print(f"\nAll {len(tests)} passed") -run([("delta_space_blend_aligns", test_delta_space_blend_aligns_with_average)]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([("delta_space_blend_aligns", test_delta_space_blend_aligns_with_average)]) diff --git a/tests/test_interp_integration.py b/tests/test_interp_integration.py index 8f76a89..ffb902b 100644 --- a/tests/test_interp_integration.py +++ b/tests/test_interp_integration.py @@ -91,8 +91,12 @@ def run(tests): print(f"\nAll {len(tests)} passed") -run([ - ("each_mode_nonzero", test_each_mode_produces_nonzero_lora), - ("additive_stronger_than_average", test_additive_stronger_than_average), - ("single_owner_not_zero", test_single_owner_key_not_zero), -]) \ No newline at end of file +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run([ + ("each_mode_nonzero", test_each_mode_produces_nonzero_lora), + ("additive_stronger_than_average", test_additive_stronger_than_average), + ("single_owner_not_zero", test_single_owner_key_not_zero), + ]) diff --git a/tests/test_merge_node_names.py b/tests/test_merge_node_names.py index 473d33c..df301eb 100644 --- a/tests/test_merge_node_names.py +++ b/tests/test_merge_node_names.py @@ -119,4 +119,8 @@ def run(): print(f"\nAll 6 passed") -run() +# Runnable as a plain script (`python tests/.py`); under pytest the +# test_* functions are collected directly, so the script runner must not fire +# at import time -- it calls sys.exit() and would abort collection. +if __name__ == "__main__": + run() diff --git a/tests/test_types.py b/tests/test_types.py index bd82f07..e48d42c 100644 --- a/tests/test_types.py +++ b/tests/test_types.py @@ -6,13 +6,10 @@ Tests type guards, validators, and type definitions from src/types.py. import pytest import torch -import sys -from pathlib import Path -# Add src to path for imports -sys.path.insert(0, str(Path(__file__).parent.parent / "src")) - -from types import ( +# Imported as `src.types`, never bare `types` -- that name is the stdlib module. +# conftest.py puts the project root on sys.path and mocks the ComfyUI modules. +from src.types import ( is_lora_tensors, is_lora_stack, validate_lora_tensors, diff --git a/tests/test_validation.py b/tests/test_validation.py index c98ec17..4f234b5 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -6,14 +6,11 @@ Tests validators for LoRA stacks, tensor shapes, and merge parameters. import pytest import torch -import sys -from pathlib import Path from unittest.mock import Mock -# Add src to path for imports -sys.path.insert(0, str(Path(__file__).parent.parent / "src")) - -from validation import ( +# Imported as `src.*` so the package-relative imports inside src/ resolve. +# conftest.py puts the project root on sys.path and mocks the ComfyUI modules. +from src.validation import ( LoRAStackValidator, TensorShapeValidator, MergeParameterValidator,