James VeitchandClaude Sonnet 5
ecfc7eca59
uv run pytest was failing every single test with ModuleNotFoundError: No module named 'comfydv._llm' — reproduces identically on a clean checkout, unrelated to any test content. Root cause: the repo root's own __init__.py (ComfyUI's custom-node entry point) is also a valid "comfydv" package from a sys.path state pytest transiently constructs during fixture setup, and it doesn't expose the _llm submodule. A bare `import comfydv` issued from inside a fixture body (e.g. _clear_ollama_caches) could resolve to that root package instead of src/comfydv. Forcing an explicit `import comfydv` in pytest_configure — while our sys.path.insert(0, src) is still the definitive answer — caches the correct module in sys.modules before anything else gets a chance to resolve it ambiguously. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YArD9ZjBWKsAvazmS48amA
Tests
This directory contains comprehensive pytest tests for the comfydv package.
Running Tests
# Run all tests
uv run pytest
# Run with verbose output
uv run pytest -v
# Run with coverage report
uv run pytest --cov=src/comfydv --cov-report=html
# Run specific test file
uv run pytest tests/test_format_string.py
# Run specific test class
uv run pytest tests/test_format_string.py::TestVariableExtraction
# Run specific test
uv run pytest tests/test_format_string.py::TestVariableExtraction::test_extract_simple_single_variable
# Run tests matching a pattern
uv run pytest -k "jinja2"
Coverage reports are available in htmlcov/index.html after running with --cov-report=html.
Test Coverage
Current: 75% (203 statements total, 51 missed)
format_string.py: 78% coverage__init__.py: 100% coveragecircuit_breaker.py: 68% coveragerandom_choice.py: 60% coverageutils.py: 75% coverage
Test Structure
conftest.py
Contains pytest configuration, fixtures, and mocks for ComfyUI dependencies:
- Mock ComfyUI modules (
comfy,server,folder_paths,aiohttp) - Uses
pytest_configurehook to install mocks before test collection - Provides
format_string_classfixture usingimportlibto directly load module - Fixtures for test data and class instances
- Pytest hooks for early mock installation
test_format_string.py
Comprehensive test suite for the FormatString node with 47 tests organized into classes:
- TestVariableExtraction (12 tests): Variable extraction from templates
- TestSimpleFormatting (4 tests): Python format string rendering
- TestJinja2Formatting (5 tests): Jinja2 template rendering
- TestDynamicOutputs (6 tests): Dynamic output configuration
- TestOutputConsistency (3 tests): Outputs match RETURN_TYPES/RETURN_NAMES
- TestInputTypes (4 tests): INPUT_TYPES method validation
- TestIsChanged (5 tests): Cache invalidation logic
- TestStatePersistence (2 tests): State saving/loading
- TestEdgeCases (4 tests): Error handling and edge cases
- TestTimeNowFunction (2 tests): time_now utility function
Mocking Strategy
The tests use importlib.util to directly load the format_string.py module, bypassing the package __init__.py which has ComfyUI dependencies. This allows testing without ComfyUI installation while maintaining the root __init__.py for ComfyUI extension discovery.
Writing Tests
Example test
def test_new_feature(self, format_string_class, sample_data):
"""Test that new feature works correctly."""
result = format_string_class.some_method(sample_data["name"])
assert result == expected_value
Available fixtures
format_string_class: Fresh FormatString class with reset statesample_templates: Dictionary of sample template stringssample_data: Dictionary of sample data for templates
Test naming convention
Use descriptive names: test_<what>_<condition>_<expected>
Troubleshooting
Debugging failed tests
# Run with verbose output
uv run pytest -vv --tb=long
# Run with pdb debugger
uv run pytest --pdb
Test isolation
Each test is independent. The format_string_class fixture provides a fresh instance with reset state.
Future Improvements
- Add integration tests with actual ComfyUI installation
- Add JavaScript tests for frontend functionality
- Increase coverage for circuit_breaker and random_choice nodes
- Add property-based testing with Hypothesis