test: run the whole suite without a ComfyUI installation

`pytest tests/` previously crashed during collection and 7 of 17 test files
were dead: 61 tests were reachable, all via ad-hoc standalone scripts. Now a
bare `pytest` collects everything and passes 238 tests with ComfyUI absent
(verified by running the suite from outside the ComfyUI tree, where
`import comfy` raises ModuleNotFoundError).

Import structure:
- Drop tests/__init__.py. With it, pytest walks up to the project root's
  __init__.py -- the ComfyUI node entry point -- and imports ComfyUI before
  any test runs.
- Import project code as `src.<module>` instead of putting src/ on sys.path
  and importing bare `merge.algorithms` / `validation` / `types`. Modules in
  src/ use package-relative imports (`from ..types import ...`) that cannot
  resolve when loaded top-level, and `types` collided with the stdlib module.
  Same change for the mock.patch targets in test_algorithms.
- Consolidate conftest.py in tests/, mocking comfy, folder_paths,
  comfy_extras and nodes. It stays in tests/ rather than the project root
  because pytest imports a root-level conftest as part of the root package,
  executing the ComfyUI entry point.
- Guard the script-style runners behind `if __name__ == "__main__":` so they
  no longer sys.exit() during collection. Those files still run standalone.
- Drop run_pytest.py: a mocking wrapper made redundant by conftest, unused
  and pointing at an unresolvable default path.

Bugs the dead tests were hiding:
- validators: the INCOMPATIBLE_DIMENSIONS check sat after the `continue` that
  skips the reference tensor, so a lone LoRA with mismatched up/down ranks
  passed validation unchecked. It is a per-LoRA check and now runs for every
  entry.
- decomposition: __init__ exported a QRDecomposer that exists nowhere, so
  `import src.decomposition` raised ImportError. Export and tests removed.

Stale expectations corrected:
- return_statistics is a constructor argument, not a decompose() kwarg.
- The zero-matrix rank guard only applies under dynamic rank selection; the
  test now exercises that path, plus a new case pinning fixed-rank behavior.
- `reconstruction_error < 0.5` for a rank-10 truncation of a random 100x50
  Gaussian is unreachable -- the optimum is 0.7557 and the decomposer hits
  0.7568. Assert near-optimality instead, and add a genuinely low-rank case
  that reconstructs to 0.003.
- sym/asym distributions differ only by float32 rounding (~5e-7), below the
  default atol of 1e-8.

RUN_TESTS.md is rewritten against the real setup: correct interpreter path,
the two test-file styles, the import rules for adding tests, and a per-file
coverage table. It no longer documents test_gradient_analyzer_integration.py,
which is not in the repo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
larsupb
2026-08-02 11:41:31 +02:00
co-authored by Claude Opus 5
parent 1ca216dcbb
commit 7973203f38
19 changed files with 352 additions and 424 deletions
+116 -212
View File
@@ -1,122 +1,147 @@
# Running Tests
Quick guide to running tests for the LoRA-Merger-ComfyUI custom nodes.
The test suite runs **without a ComfyUI installation**. ComfyUI's modules
(`comfy`, `folder_paths`, `comfy_extras`, `nodes`) are mocked in
[tests/conftest.py](tests/conftest.py), so the only requirements are torch,
mergekit and pytest.
## Integration Test
### Quick Start
## Quick Start
```bash
cd /home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py
/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m pytest tests/
```
### What It Tests
Expected: **238 passed**.
The integration test validates the **PM LoRA Semantic Analyzer (Gradient)** pipeline:
Any Python with the dependencies below works; `/home/lars/SD/Apps/ComfyUI/.venv`
is simply the environment that already has them on this machine.
1. ✅ Dependencies (PyTorch, diffusers, safetensors)
2. ✅ Node instantiation (all semantic merge nodes)
3. ✅ INPUT_TYPES configuration
4. ✅ Checkpoint dropdown population
5. ✅ Specification parsing
### Expected Output
```
================================================================================
PM LoRA Semantic Analyzer (Gradient) - Integration Test
================================================================================
[... file checks ...]
================================================================================
TEST SUMMARY
================================================================================
✓ PASS Dependencies
✓ PASS LoRA Power Stacker
✓ PASS Semantic Analyzer Instantiation
✓ PASS Semantic Merge Spec
✓ PASS Semantic Merger Instantiation
Results: 5/5 tests passed
🎉 ALL TESTS PASSED!
```
### Exit Codes
- `0` - All tests passed
- `1` - One or more tests failed
- `130` - Test interrupted (Ctrl+C)
## Unit Tests
### Run All Unit Tests
### Run a single file or test
```bash
cd /home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/
PY=/home/lars/SD/Apps/ComfyUI/.venv/bin/python
$PY -m pytest tests/test_validation.py # one file
$PY -m pytest tests/test_gta_sparsify.py -v # verbose
$PY -m pytest tests/test_types.py::TestTypeGuards # one class
$PY -m pytest tests/ -k "sparsify or interp" # by name
```
### Run Specific Test Files
### Coverage
```bash
# Test types system
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_types.py -v
# Test decomposition
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_decomposition.py -v
# Test validation
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_validation.py -v
# Test algorithms
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest tests/test_algorithms.py -v
/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m pytest --cov=src tests/
```
### Run with Coverage
Requires `pytest-cov` (see `requirements-dev.txt`).
## Two Kinds of Test File
Both kinds are collected by `pytest tests/`; the difference only matters when
you want to run one directly.
**Script-style** — plain asserts driven by a `run([...])` list, guarded behind
`if __name__ == "__main__":`. These also run as standalone scripts with no
pytest at all, which is useful for a quick check or when bisecting:
```bash
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pytest --cov=src tests/
/home/lars/SD/Apps/ComfyUI/.venv/bin/python tests/test_gta_sparsify.py
# -> PASS magnitude ... All 12 passed
```
## Syntax Validation
Files: `test_gta_behavior`, `test_gta_merge`, `test_gta_parity`,
`test_gta_sparsify`, `test_interp_delta_merge`, `test_interp_fidelity`,
`test_interp_integration`, `test_lora_save`, `test_merge_node_names`,
`test_merger_vram_offload`.
### Check Python Syntax
**Pytest-style** — `Test*` classes, fixtures, `pytest.raises`. Run these through
pytest only; executing them directly does not set up the import paths.
```bash
# Single file
python3 -m py_compile src/nodes_semantic_merge.py
Files: `test_algorithms`, `test_blocks`, `test_decomposition`,
`test_spectral_norm`, `test_types`, `test_utility`, `test_validation`.
# All Python files
find . -name "*.py" -not -path "./venv/*" -exec python3 -m py_compile {} \;
```
## How Imports Are Set Up
### Check with Flake8
Two rules keep the suite importable outside ComfyUI. Both are handled by
[tests/conftest.py](tests/conftest.py); they matter when you add a test.
```bash
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m flake8 src/ --max-line-length=120
```
1. **Import project code as `src.<module>`**, never as a bare top-level module:
```python
from src.validation import LoRAStackValidator # correct
from validation import LoRAStackValidator # breaks
from types import is_lora_tensors # breaks: stdlib `types`
```
Modules under `src/` use package-relative imports (`from ..types import ...`),
which only resolve when `src` itself is the package. conftest puts the project
root on `sys.path` for this. The same applies to `unittest.mock.patch`
targets: `@patch('src.merge.algorithms.LinearMergeTask')`.
2. **Do not add `tests/__init__.py`.** With it, pytest walks up to the project
root's `__init__.py` — the ComfyUI node entry point — and imports ComfyUI
before any test runs.
`conftest.py` lives in `tests/`, not the project root, for the same reason: a
root-level conftest is imported as part of that package.
## Test Files
### Integration Tests
- `test_gradient_analyzer_integration.py` - Validates semantic analyzer pipeline
| File | Covers |
|------|--------|
| `test_algorithms.py` | Merge algorithm registry and dispatch |
| `test_blocks.py` | Block selection, key normalization, block weights |
| `test_decomposition.py` | SVD / randomized SVD / energy-based decomposers |
| `test_gta_behavior.py` | GTA delta-space merge behavior |
| `test_gta_merge.py` | Sign election and disjoint merge |
| `test_gta_parity.py` | Parity of local sparsify vs. mergekit |
| `test_gta_sparsify.py` | magnitude / outliers / bernoulli / della sparsify |
| `test_interp_delta_merge.py` | slerp / nuslerp / karcher / nearswap in delta space |
| `test_interp_fidelity.py` | Delta-space blend vs. reference average |
| `test_interp_integration.py` | End-to-end merge through the node |
| `test_lora_save.py` | Tensor sanitization before safetensors save |
| `test_merge_node_names.py` | Node widget names and titles |
| `test_merger_vram_offload.py` | `offload_models` widget behavior |
| `test_spectral_norm.py` | Spectral norm scaling |
| `test_types.py` | Type guards and validators |
| `test_utility.py` | SVD/QR pipeline helpers |
| `test_validation.py` | LoRA stack, shape and parameter validation |
### Unit Tests (in `tests/` directory)
- `test_types.py` - Type system and validators
- `test_algorithms.py` - Merge algorithms
- `test_decomposition.py` - SVD/QR decomposition
- `test_validation.py` - Input validation
## Requirements
### Test Workflows
- `/home/lars/SD/Apps/ComfyUI/user/default/workflows/PM-Gradient-Analyzer-zImage.json` - Example workflow
```bash
pip install -r requirements-dev.txt
```
Needed to run the suite: `torch`, `mergekit`, `pytest`. Optional:
`pytest-cov` (coverage), `pytest-xdist` (`-n auto` parallel runs).
A ComfyUI installation is **not** required.
## Syntax and Lint
```bash
find src -name "*.py" -exec python3 -m py_compile {} \;
/home/lars/SD/Apps/ComfyUI/.venv/bin/python -m flake8 src/ --max-line-length=120
```
## Debugging Failures
```bash
PY=/home/lars/SD/Apps/ComfyUI/.venv/bin/python
$PY -m pytest tests/ -vv --tb=long # full tracebacks
$PY -m pytest tests/ -x # stop at first failure
$PY -m pytest tests/ --lf # rerun last failures
```
If a new test fails at import with `attempted relative import beyond top-level
package`, `No module named 'src'`, or picks up the stdlib `types` module, it is
importing project code as a top-level module — see "How Imports Are Set Up".
## Continuous Integration
### GitHub Actions Example
```yaml
name: Tests
@@ -126,138 +151,17 @@ jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v2
- uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: '3.11'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install pytest pytest-cov
run: pip install -r requirements-dev.txt
- name: Run integration tests
run: python3 test_gradient_analyzer_integration.py
- name: Run unit tests
run: pytest tests/ -v --cov=src
- name: Run tests
run: pytest tests/ -v
```
## Test Requirements
### For Integration Tests
**Required:**
- ComfyUI installed at `/home/lars/SD/Apps/ComfyUI`
- venv at `/home/lars/SD/Apps/ComfyUI/venv`
- Dependencies: `torch`, `diffusers>=0.30.0`, `safetensors`
**Optional (for file checks):**
- zImage checkpoint in `models/diffusion_models/`
- LoRAs in `models/loras/`
### For Unit Tests
**Required:**
- PyTorch
- Standard Python libraries
**Optional:**
- pytest
- pytest-cov (for coverage reports)
## Quick Commands Reference
```bash
# Integration test (basic validation)
./test_gradient_analyzer_integration.py
# Unit tests (comprehensive)
pytest tests/ -v
# Syntax check
python3 -m py_compile src/*.py
# Coverage report
pytest --cov=src --cov-report=html tests/
# Specific test
pytest tests/test_types.py::test_validate_lora_tensors -v
```
## Debugging Test Failures
### Enable Verbose Output
```bash
# Integration test
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py 2>&1 | tee test_output.log
# Unit tests
pytest tests/ -vv --tb=long
```
### Check Imports
```bash
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c "
import sys
sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI')
sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI/custom_nodes/LoRA-Merger-ComfyUI')
from src.nodes_semantic_merge import PMLoRASemanticAnalyzerGradient
print('✓ Import successful')
"
```
### Test Individual Components
```bash
# Test diffusers
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c "
from diffusers.models import ZImageTransformer2DModel
print('✓ ZImageTransformer2DModel available')
"
# Test folder_paths
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -c "
import sys
sys.path.insert(0, '/home/lars/SD/Apps/ComfyUI')
import folder_paths
print('✓ folder_paths available')
"
```
## Performance Testing
### Time Individual Tests
```bash
time /home/lars/SD/Apps/ComfyUI/venv/bin/python3 test_gradient_analyzer_integration.py
```
### Profile Tests
```bash
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m cProfile -o test_profile.prof test_gradient_analyzer_integration.py
# View profile
/home/lars/SD/Apps/ComfyUI/venv/bin/python3 -m pstats test_profile.prof
```
## Documentation
- **TEST_GRADIENT_ANALYZER.md** - Detailed test documentation
- **QUICK_START_GRADIENT_ANALYZER.md** - User guide
- **ZIMAGE_LOADER_OPTIMIZATION.md** - Technical details
- **BUGFIX_ZIMAGE_IMPORT.md** - Import fix documentation
## Support
For issues or questions:
1. Check test output for error messages
2. Review documentation files
3. Verify all dependencies are installed
4. Check that files exist in expected locations
5. Open an issue on GitHub if problems persist
No ComfyUI checkout is needed in CI.
-29
View File
@@ -1,29 +0,0 @@
#!/usr/bin/env python3
"""
Test runner that sets up mocking before pytest runs
"""
import sys
from unittest.mock import MagicMock
from typing import Tuple
import torch
# Mock ComfyUI modules BEFORE any imports
sys.modules['comfy'] = MagicMock()
sys.modules['comfy.utils'] = MagicMock()
sys.modules['comfy.model_management'] = MagicMock()
sys.modules['comfy.lora'] = MagicMock()
# Mock architectures module
architectures_mock = MagicMock()
architectures_mock.sd_lora.UP_DOWN_ALPHA_TUPLE = Tuple[torch.Tensor, torch.Tensor, float]
sys.modules['architectures'] = architectures_mock
sys.modules['architectures.sd_lora'] = architectures_mock.sd_lora
sys.modules['architectures.general_architecture'] = MagicMock()
sys.modules['architectures.wan_lora'] = MagicMock()
# Now run pytest
import pytest
if __name__ == "__main__":
sys.exit(pytest.main(sys.argv[1:] or ["test_utility.py", "-v"]))
-2
View File
@@ -16,7 +16,6 @@ from .svd import (
SVDDecomposer,
RandomizedSVDDecomposer,
EnergyBasedRandomizedSVDDecomposer,
QRDecomposer,
)
__all__ = [
@@ -28,5 +27,4 @@ __all__ = [
'SVDDecomposer',
'RandomizedSVDDecomposer',
'EnergyBasedRandomizedSVDDecomposer',
'QRDecomposer',
]
+18 -15
View File
@@ -138,6 +138,24 @@ class TensorShapeValidator:
# Check all tensors against reference
for lora_name, (up, down, alpha) in tensors_dict.items():
# Validate up/down dimensions are compatible. This is a per-LoRA
# (self-consistency) check, so it runs for the reference tensor too --
# otherwise a single incompatible LoRA passes validation unexamined.
if len(up.shape) >= 2 and len(down.shape) >= 2:
up_rank = up.shape[1]
down_rank = down.shape[0]
if up_rank != down_rank:
errors.append({
"code": "INCOMPATIBLE_DIMENSIONS",
"message": (
f"LoRA '{lora_name}' has incompatible up/down dimensions: "
f"up rank {up_rank} != down rank {down_rank}"
),
"location": lora_name
})
# The remaining checks compare against the reference tensor.
if lora_name == first_name:
continue
@@ -165,21 +183,6 @@ class TensorShapeValidator:
f"differs from reference {ref_down_shape}"
)
# Validate up/down dimensions are compatible
if len(up.shape) >= 2 and len(down.shape) >= 2:
up_rank = up.shape[1] if len(up.shape) == 2 else up.shape[1]
down_rank = down.shape[0]
if up_rank != down_rank:
errors.append({
"code": "INCOMPATIBLE_DIMENSIONS",
"message": (
f"LoRA '{lora_name}' has incompatible up/down dimensions: "
f"up rank {up_rank} != down rank {down_rank}"
),
"location": lora_name
})
return {
"valid": len(errors) == 0,
"errors": errors,
-6
View File
@@ -1,6 +0,0 @@
"""
Test suite for LoRA Power-Merger.
Unit tests for core functionality. Integration tests are not included
as they require a running ComfyUI instance.
"""
+78 -49
View File
@@ -1,80 +1,109 @@
"""
Pytest configuration and fixtures for LoRA Power-Merger tests
"""Pytest configuration for LoRA Power-Merger.
This file sets up the test environment by mocking ComfyUI dependencies
that may not be available in the test environment.
Sets up the test environment so the whole suite runs on torch, mergekit and
pytest alone -- no ComfyUI installation required.
Deliberately lives in tests/ rather than at the project root: the root holds an
``__init__.py`` (the ComfyUI node entry point, which imports ComfyUI), so pytest
would import a root-level conftest as a submodule of that package and execute the
entry point before any test ran. tests/ is not a package, so this file imports on
its own.
"""
import sys
import os
import sys
import types
from unittest.mock import MagicMock
from typing import Tuple
import torch
# Add the src directory to the path to allow imports
project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
src_path = os.path.join(project_root, 'src')
if src_path not in sys.path:
sys.path.insert(0, src_path)
# Put the PROJECT ROOT (not src/) on the path, so modules under test are imported
# as `src.<module>`. Modules in src/ use package-relative imports (`from ..types
# import ...`); importing them as top-level modules makes those resolve beyond
# the top-level package and fail. src/__init__.py is empty, so this pulls in no
# ComfyUI dependencies by itself.
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if PROJECT_ROOT not in sys.path:
sys.path.insert(0, PROJECT_ROOT)
def _stub_root_package():
"""Stop pytest from executing the project root's ``__init__.py``.
Because the root holds an ``__init__.py``, pytest treats the project root as a
package and imports it in ``Package.setup()`` before running any test under
it. That file is the ComfyUI node entry point (``from .src.lora_apply import
...``), so importing it outside a ComfyUI process fails on the relative import
-- and it is not test code in any case.
``Package.setup()`` goes through ``import_path``, which under the ``importlib``
import mode returns an already-present ``sys.modules`` entry before touching
the file. The module name it derives for ``<rootdir>/__init__.py`` is
``"__init__"``; seeding that name with an empty module makes the import a
no-op. Registered here rather than via ``pytest_collect_directory``: that hook
fires only while recursing into directories, not for the parent packages of a
test path named directly on the command line.
"""
sys.modules.setdefault("__init__", types.ModuleType("__init__"))
# Prevent pytest from treating the project root as a package
# by ensuring tests are collected from tests directory only
import pytest
def pytest_ignore_collect(collection_path, config):
"""Ignore collection of __init__.py files and src/ directory."""
"""Never collect package __init__.py files or anything under src/."""
path_str = str(collection_path)
if path_str.endswith('__init__.py'):
return True
if '/src/' in path_str or path_str.endswith('/src'):
if (os.sep + 'src' + os.sep) in path_str or path_str.endswith(os.sep + 'src'):
return True
return False
# Mock ComfyUI modules before any test imports
def pytest_configure(config):
"""Configure pytest and mock unavailable modules"""
# Mock comfy module
def _mock_comfyui_modules():
"""Install stand-ins for the ComfyUI modules src/ imports at module scope."""
comfy_mock = MagicMock()
sys.modules['comfy'] = comfy_mock
sys.modules['comfy.utils'] = MagicMock()
sys.modules['comfy.model_management'] = MagicMock()
sys.modules['comfy.lora'] = MagicMock()
sys.modules['comfy.weight_adapter'] = MagicMock()
sys.modules['comfy.model_patcher'] = MagicMock()
sys.modules['comfy.sd'] = MagicMock()
for name in ('utils', 'model_management', 'lora', 'weight_adapter',
'model_patcher', 'sd'):
sys.modules[f'comfy.{name}'] = MagicMock()
# Create a LoRAAdapter mock class
class LoRAAdapterMock:
def __init__(self, *args, **kwargs):
self.state_dict = {}
"""Stand-in for comfy.weight_adapter.LoRAAdapter.
Mirrors the real signature: it is a plain container over ``loaded_keys``
and a ``weights`` 6-tuple (up, down, alpha, mid, dora_scale, reshape),
which the merger reads back after building its output state dict.
"""
name = "lora"
def __init__(self, loaded_keys=None, weights=None):
self.loaded_keys = set() if loaded_keys is None else loaded_keys
self.weights = weights
comfy_mock.weight_adapter.LoRAAdapter = LoRAAdapterMock
sys.modules['comfy.weight_adapter'].LoRAAdapter = LoRAAdapterMock
# Mock comfy_util module
comfy_util_mock = MagicMock()
sys.modules['comfy_util'] = comfy_util_mock
# The merger logs free VRAM around the offload step, formatting the result
# with `:.0f` -- a bare MagicMock would raise on __format__.
mm = sys.modules['comfy.model_management']
mm.get_free_memory.return_value = 8 * 1024 ** 3
comfy_mock.model_management = mm
comfy_mock.utils = sys.modules['comfy.utils']
# Mock nodes module (ComfyUI nodes)
nodes_mock = MagicMock()
sys.modules['nodes'] = nodes_mock
sys.modules['comfy_extras'] = MagicMock()
sys.modules['comfy_extras.nodes_custom_sampler'] = MagicMock()
sys.modules['nodes'] = MagicMock()
# Mock folder_paths module (ComfyUI utility)
folder_paths_mock = MagicMock()
folder_paths_mock.get_folder_paths.return_value = []
folder_paths_mock.folder_names_and_paths = {}
sys.modules['folder_paths'] = folder_paths_mock
# Mock comfy_extras module (ComfyUI extras)
comfy_extras_mock = MagicMock()
sys.modules['comfy_extras'] = comfy_extras_mock
sys.modules['comfy_extras.nodes_custom_sampler'] = MagicMock()
# Mock architectures module to avoid relative import issues
architectures_mock = MagicMock()
# Define the UP_DOWN_ALPHA_TUPLE type alias
architectures_mock.sd_lora.UP_DOWN_ALPHA_TUPLE = Tuple[torch.Tensor, torch.Tensor, float]
sys.modules['architectures'] = architectures_mock
sys.modules['architectures.sd_lora'] = architectures_mock.sd_lora
sys.modules['architectures.general_architecture'] = MagicMock()
sys.modules['architectures.wan_lora'] = MagicMock()
def pytest_configure(config):
"""Mock ComfyUI before test modules (and the src/ modules they import) load."""
_mock_comfyui_modules()
_stub_root_package()
# Also applied at import time: conftest is imported before collection, so this
# covers anything that resolves earlier than pytest_configure.
_mock_comfyui_modules()
_stub_root_package()
+6 -9
View File
@@ -7,19 +7,16 @@ Mock mergekit dependencies to test algorithm logic in isolation.
import pytest
import torch
import sys
from pathlib import Path
from unittest.mock import Mock, MagicMock, patch
# Add src to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from merge.algorithms import (
# Imported as `src.*` so the package-relative imports inside src/ resolve.
# conftest.py puts the project root on sys.path and mocks the ComfyUI modules.
from src.merge.algorithms import (
linear_merge,
get_merge_algorithm,
MERGE_ALGORITHMS,
)
from merge.utils import apply_weights_to_tensors
from src.merge.utils import apply_weights_to_tensors
class TestApplyWeightsToTensors:
@@ -116,7 +113,7 @@ class TestAlgorithmRegistry:
class TestLinearMerge:
"""Tests for linear merge algorithm."""
@patch('merge.algorithms.LinearMergeTask')
@patch('src.merge.algorithms.LinearMergeTask')
def test_linear_merge_calls_task(self, mock_task_class):
"""Test that linear merge creates and executes LinearMergeTask."""
# Setup mocks
@@ -153,7 +150,7 @@ class TestLinearMerge:
# Verify result
assert torch.allclose(result, torch.ones(10, 10))
@patch('merge.algorithms.LinearMergeTask')
@patch('src.merge.algorithms.LinearMergeTask')
def test_linear_merge_default_normalize(self, mock_task_class):
"""Test that normalize defaults to False if not in method_args."""
mock_task = Mock()
+5 -5
View File
@@ -1,4 +1,4 @@
from blocks import normalize_key, parse_weight_list
from src.blocks import normalize_key, parse_weight_list
class TestNormalizeKey:
@@ -31,7 +31,7 @@ class TestParseWeightList:
assert parse_weight_list(None) == []
from blocks import make_category, key_weight
from src.blocks import make_category, key_weight
KREA2_DEF = {
"model": "KREA2",
@@ -85,7 +85,7 @@ class TestKeyWeight:
from collections import OrderedDict
from blocks import compute_lora_weights, merge_selection, build_block_selection_dict, resolve_block_selection
from src.blocks import compute_lora_weights, merge_selection, build_block_selection_dict, resolve_block_selection
class TestComputeLoraWeights:
@@ -186,7 +186,7 @@ class TestResolveBlockSelection:
import torch
from blocks import apply_block_weights
from src.blocks import apply_block_weights
class TestApplyBlockWeights:
@@ -222,7 +222,7 @@ class TestApplyBlockWeights:
assert torch.allclose(up, torch.ones(4, 2))
from blocks import build_krea2_definition, build_klein_definition
from src.blocks import build_krea2_definition, build_klein_definition
class TestBuildKrea2:
+46 -40
View File
@@ -6,17 +6,13 @@ Tests tensor decomposition functionality including SVD, QR, and error handling.
import pytest
import torch
import sys
from pathlib import Path
# Add src to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from decomposition import (
# Imported as `src.*` so the package-relative imports inside src/ resolve.
# conftest.py puts the project root on sys.path and mocks the ComfyUI modules.
from src.decomposition import (
SVDDecomposer,
RandomizedSVDDecomposer,
EnergyBasedRandomizedSVDDecomposer,
QRDecomposer,
SingularValueDistribution,
)
@@ -26,23 +22,39 @@ class TestSVDDecomposer:
def test_basic_2d_decomposition(self):
"""Test basic 2D tensor decomposition."""
decomposer = SVDDecomposer()
# return_statistics is a constructor option, not a decompose() argument.
decomposer = SVDDecomposer(return_statistics=True)
weight = torch.randn(100, 50)
target_rank = 10
up, down, alpha, stats = decomposer.decompose(
weight, target_rank, return_statistics=True
)
up, down, alpha, stats = decomposer.decompose(weight, target_rank)
# Check shapes
assert up.shape == (100, 10)
assert down.shape == (10, 50)
assert isinstance(alpha, float)
# Check reconstruction quality
# Reconstruction quality is bounded by the discarded singular values, not
# by the decomposer: for a full-rank random Gaussian, a rank-10 truncation
# of 50 singular values necessarily loses ~75% of the Frobenius norm.
# Assert we land at that optimum rather than at an arbitrary threshold.
reconstructed = up @ down
reconstruction_error = torch.norm(weight - reconstructed) / torch.norm(weight)
assert reconstruction_error < 0.5 # Should be reasonable approximation
S = torch.linalg.svdvals(weight)
optimal_error = (S[target_rank:].pow(2).sum() / S.pow(2).sum()).sqrt()
assert reconstruction_error < optimal_error + 0.01
def test_low_rank_input_reconstructs_accurately(self):
"""A genuinely low-rank matrix is recovered with little error."""
decomposer = SVDDecomposer()
weight = torch.randn(100, 10) @ torch.randn(10, 50)
weight = weight + 0.01 * torch.randn(100, 50)
up, down, alpha, _ = decomposer.decompose(weight, target_rank=10)
reconstruction_error = torch.norm(weight - up @ down) / torch.norm(weight)
assert reconstruction_error < 0.05
def test_4d_conv_decomposition(self):
"""Test 4D convolutional tensor decomposition."""
@@ -77,7 +89,11 @@ class TestSVDDecomposer:
recon_sym = up_sym @ down_sym
recon_asym = up_asym @ down_asym
assert torch.allclose(recon_sym, recon_asym, rtol=1e-4)
# The product is mathematically identical either way -- the distributions
# only decide whether S goes into up, into down, or is split as sqrt(S)
# across both. atol is needed because the default (1e-8) is below float32
# rounding for entries near zero; observed difference is ~5e-7.
assert torch.allclose(recon_sym, recon_asym, rtol=1e-4, atol=1e-5)
def test_statistics_calculation(self):
"""Test that statistics are calculated correctly."""
@@ -211,27 +227,6 @@ class TestEnergyBasedRandomizedSVDDecomposer:
assert stats_high is not None
class TestQRDecomposer:
"""Tests for QR decomposer."""
def test_qr_decomposition(self):
"""Test basic QR decomposition."""
decomposer = QRDecomposer()
weight = torch.randn(100, 50)
target_rank = 20
up, down, alpha, _ = decomposer.decompose(weight, target_rank)
# Check shapes
assert up.shape == (100, 20)
assert down.shape == (20, 50)
# QR decomposition should still provide reasonable approximation
reconstructed = up @ down
reconstruction_error = torch.norm(weight - reconstructed) / torch.norm(weight)
assert reconstruction_error < 1.0 # Looser bound than SVD
class TestErrorHandling:
"""Tests for error handling in decomposition."""
@@ -248,11 +243,24 @@ class TestErrorHandling:
decomposer = SVDDecomposer(return_statistics=True)
weight = torch.zeros(50, 30)
# Should not crash, should handle gracefully
# The zero-matrix guard lives in dynamic rank selection; without a
# dynamic_method the caller has pinned the rank and target_rank is honoured.
up, down, alpha, stats = decomposer.decompose(
weight, target_rank=10, dynamic_method="sv_ratio", dynamic_param=2.0
)
# Rank should collapse to the minimum for a numerically zero matrix
assert stats['new_rank'] == 1
def test_zero_matrix_fixed_rank_is_honoured(self):
"""A pinned rank (no dynamic_method) is kept even for a zero matrix."""
decomposer = SVDDecomposer(return_statistics=True)
weight = torch.zeros(50, 30)
up, down, alpha, stats = decomposer.decompose(weight, target_rank=10)
# Rank should be minimal for zero matrix
assert stats['new_rank'] == 1
assert stats['new_rank'] == 10
assert torch.allclose(up @ down, torch.zeros(50, 30))
def test_invalid_dynamic_method(self):
"""Test that invalid dynamic method raises error."""
@@ -290,7 +298,6 @@ class TestIntegrationWithFixtures:
SVDDecomposer(),
RandomizedSVDDecomposer(),
EnergyBasedRandomizedSVDDecomposer(),
QRDecomposer(),
]
for decomposer in decomposers:
@@ -309,7 +316,6 @@ class TestIntegrationWithFixtures:
decomposers = [
SVDDecomposer(),
RandomizedSVDDecomposer(),
QRDecomposer(),
]
for decomposer in decomposers:
+9 -5
View File
@@ -57,8 +57,12 @@ def test_normalize_does_not_collapse_as_1_over_n_squared():
assert merged.norm() > 0.5 * avg.norm()
run([
("refactor_reconstructs", test_refactor_reconstructs_delta),
("nonoverlap_keeps_strength", test_style_plus_character_nonoverlap_keeps_strength),
("no_1_over_n_squared", test_normalize_does_not_collapse_as_1_over_n_squared),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("refactor_reconstructs", test_refactor_reconstructs_delta),
("nonoverlap_keeps_strength", test_style_plus_character_nonoverlap_keeps_strength),
("no_1_over_n_squared", test_normalize_does_not_collapse_as_1_over_n_squared),
])
+11 -7
View File
@@ -48,10 +48,14 @@ def test_n_equal_loras_scale_one_over_n_not_squared():
assert torch.allclose(merged, d)
run([
("elect_sign", test_elect_sign_weighted_majority),
("nonoverlap_keeps_strength", test_disjoint_normalize_nonoverlap_keeps_strength),
("conflict_arbitrated", test_disjoint_normalize_overlap_conflict_arbitrated),
("linear_weighted_average", test_linear_normalize_is_weighted_average),
("n_loras_1_over_n", test_n_equal_loras_scale_one_over_n_not_squared),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("elect_sign", test_elect_sign_weighted_majority),
("nonoverlap_keeps_strength", test_disjoint_normalize_nonoverlap_keeps_strength),
("conflict_arbitrated", test_disjoint_normalize_overlap_conflict_arbitrated),
("linear_weighted_average", test_linear_normalize_is_weighted_average),
("n_loras_1_over_n", test_n_equal_loras_scale_one_over_n_not_squared),
])
+9 -5
View File
@@ -63,8 +63,12 @@ def test_resolve_rescale_default_matches_mergekit():
assert gta.resolve_rescale_norm("ties", "l2") == "l2"
run([
("task_arithmetic_parity", test_task_arithmetic_parity),
("ties_parity", test_ties_parity),
("resolve_rescale_default_matches_mergekit", test_resolve_rescale_default_matches_mergekit),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("task_arithmetic_parity", test_task_arithmetic_parity),
("ties_parity", test_ties_parity),
("resolve_rescale_default_matches_mergekit", test_resolve_rescale_default_matches_mergekit),
])
+18 -14
View File
@@ -138,17 +138,21 @@ def test_della_wide_layer_preserves_density_and_monotonic():
assert kept_mask[hi].float().mean().item() > kept_mask[~hi].float().mean().item() + 0.05
run([
("magnitude", test_magnitude_matches_mergekit),
("magnitude_rescale_l1", test_magnitude_rescale_l1_matches),
("outliers", test_outliers_matches_mergekit),
("bernoulli_seed", test_bernoulli_matches_with_seed),
("della_seed", test_della_matches_with_seed),
("density_one_identity", test_density_one_is_identity),
("della_large_density", test_della_large_preserves_density),
("della_large_l1", test_della_large_l1_preserved),
("della_large_monotonic", test_della_large_keep_prob_monotonic_in_magnitude),
("della_chunk_density", test_della_chunk_boundary_invariant_density),
("della_scatter_rank_equivalence", test_della_scatter_rank_equivalence),
("della_wide_density_monotonic", test_della_wide_layer_preserves_density_and_monotonic),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("magnitude", test_magnitude_matches_mergekit),
("magnitude_rescale_l1", test_magnitude_rescale_l1_matches),
("outliers", test_outliers_matches_mergekit),
("bernoulli_seed", test_bernoulli_matches_with_seed),
("della_seed", test_della_matches_with_seed),
("density_one_identity", test_density_one_is_identity),
("della_large_density", test_della_large_preserves_density),
("della_large_l1", test_della_large_l1_preserved),
("della_large_monotonic", test_della_large_keep_prob_monotonic_in_magnitude),
("della_chunk_density", test_della_chunk_boundary_invariant_density),
("della_scatter_rank_equivalence", test_della_scatter_rank_equivalence),
("della_wide_density_monotonic", test_della_wide_layer_preserves_density_and_monotonic),
])
+11 -7
View File
@@ -118,10 +118,14 @@ def run(tests):
print(f"\nAll {len(tests)} passed")
run([
("modes_constant", test_modes_constant),
("additive_is_blend_times_sum", test_additive_is_blend_times_sum),
("average_is_blend_times_mean", test_average_is_blend_times_mean),
("strength_linear_gain_additive", test_strength_is_linear_gain_additive),
("single_owner_fallback", test_single_owner_fallback_not_zero),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("modes_constant", test_modes_constant),
("additive_is_blend_times_sum", test_additive_is_blend_times_sum),
("average_is_blend_times_mean", test_average_is_blend_times_mean),
("strength_linear_gain_additive", test_strength_is_linear_gain_additive),
("single_owner_fallback", test_single_owner_fallback_not_zero),
])
+5 -1
View File
@@ -61,4 +61,8 @@ def run(tests):
print(f"\nAll {len(tests)} passed")
run([("delta_space_blend_aligns", test_delta_space_blend_aligns_with_average)])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([("delta_space_blend_aligns", test_delta_space_blend_aligns_with_average)])
+9 -5
View File
@@ -91,8 +91,12 @@ def run(tests):
print(f"\nAll {len(tests)} passed")
run([
("each_mode_nonzero", test_each_mode_produces_nonzero_lora),
("additive_stronger_than_average", test_additive_stronger_than_average),
("single_owner_not_zero", test_single_owner_key_not_zero),
])
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run([
("each_mode_nonzero", test_each_mode_produces_nonzero_lora),
("additive_stronger_than_average", test_additive_stronger_than_average),
("single_owner_not_zero", test_single_owner_key_not_zero),
])
+5 -1
View File
@@ -119,4 +119,8 @@ def run():
print(f"\nAll 6 passed")
run()
# Runnable as a plain script (`python tests/<file>.py`); under pytest the
# test_* functions are collected directly, so the script runner must not fire
# at import time -- it calls sys.exit() and would abort collection.
if __name__ == "__main__":
run()
+3 -6
View File
@@ -6,13 +6,10 @@ Tests type guards, validators, and type definitions from src/types.py.
import pytest
import torch
import sys
from pathlib import Path
# Add src to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from types import (
# Imported as `src.types`, never bare `types` -- that name is the stdlib module.
# conftest.py puts the project root on sys.path and mocks the ComfyUI modules.
from src.types import (
is_lora_tensors,
is_lora_stack,
validate_lora_tensors,
+3 -6
View File
@@ -6,14 +6,11 @@ Tests validators for LoRA stacks, tensor shapes, and merge parameters.
import pytest
import torch
import sys
from pathlib import Path
from unittest.mock import Mock
# Add src to path for imports
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from validation import (
# Imported as `src.*` so the package-relative imports inside src/ resolve.
# conftest.py puts the project root on sys.path and mocks the ComfyUI modules.
from src.validation import (
LoRAStackValidator,
TensorShapeValidator,
MergeParameterValidator,