James VeitchandClaude Sonnet 5
e534fac121
FormatString._extract_keys used hand-rolled regexes to detect variables
in Jinja2 templates, which broke on anything beyond bare {{ var }} and
{{ var | filter }}:
- {% for hint in extraction_hints %}{{ hint }}{% endfor %} incorrectly
surfaced the loop-local `hint` as a required input and never detected
`extraction_hints` itself, since the control-structure regex only
matched filter syntax (word followed by `|`).
- {% if extraction_hints is defined %} never matched at all.
- Filters called with arguments, e.g. {{ x | tojson(indent=2) }}, broke
the regex's anchor to the closing `}}` and silently extracted nothing.
Replaced with jinja2.meta.find_undeclared_variables() over the parsed
AST, which handles all of Jinja2's syntax correctly and excludes names
bound within the template (loop targets, {% set %}) by construction.
Since that call returns an unordered set, sort by first textual
occurrence to keep extraction order deterministic for callers that rely
on positional outputs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YArD9ZjBWKsAvazmS48amA
Tests
This directory contains comprehensive pytest tests for the comfydv package.
Running Tests
# Run all tests
uv run pytest
# Run with verbose output
uv run pytest -v
# Run with coverage report
uv run pytest --cov=src/comfydv --cov-report=html
# Run specific test file
uv run pytest tests/test_format_string.py
# Run specific test class
uv run pytest tests/test_format_string.py::TestVariableExtraction
# Run specific test
uv run pytest tests/test_format_string.py::TestVariableExtraction::test_extract_simple_single_variable
# Run tests matching a pattern
uv run pytest -k "jinja2"
Coverage reports are available in htmlcov/index.html after running with --cov-report=html.
Test Coverage
Current: 75% (203 statements total, 51 missed)
format_string.py: 78% coverage__init__.py: 100% coveragecircuit_breaker.py: 68% coveragerandom_choice.py: 60% coverageutils.py: 75% coverage
Test Structure
conftest.py
Contains pytest configuration, fixtures, and mocks for ComfyUI dependencies:
- Mock ComfyUI modules (
comfy,server,folder_paths,aiohttp) - Uses
pytest_configurehook to install mocks before test collection - Provides
format_string_classfixture usingimportlibto directly load module - Fixtures for test data and class instances
- Pytest hooks for early mock installation
test_format_string.py
Comprehensive test suite for the FormatString node with 47 tests organized into classes:
- TestVariableExtraction (12 tests): Variable extraction from templates
- TestSimpleFormatting (4 tests): Python format string rendering
- TestJinja2Formatting (5 tests): Jinja2 template rendering
- TestDynamicOutputs (6 tests): Dynamic output configuration
- TestOutputConsistency (3 tests): Outputs match RETURN_TYPES/RETURN_NAMES
- TestInputTypes (4 tests): INPUT_TYPES method validation
- TestIsChanged (5 tests): Cache invalidation logic
- TestStatePersistence (2 tests): State saving/loading
- TestEdgeCases (4 tests): Error handling and edge cases
- TestTimeNowFunction (2 tests): time_now utility function
Mocking Strategy
The tests use importlib.util to directly load the format_string.py module, bypassing the package __init__.py which has ComfyUI dependencies. This allows testing without ComfyUI installation while maintaining the root __init__.py for ComfyUI extension discovery.
Writing Tests
Example test
def test_new_feature(self, format_string_class, sample_data):
"""Test that new feature works correctly."""
result = format_string_class.some_method(sample_data["name"])
assert result == expected_value
Available fixtures
format_string_class: Fresh FormatString class with reset statesample_templates: Dictionary of sample template stringssample_data: Dictionary of sample data for templates
Test naming convention
Use descriptive names: test_<what>_<condition>_<expected>
Troubleshooting
Debugging failed tests
# Run with verbose output
uv run pytest -vv --tb=long
# Run with pdb debugger
uv run pytest --pdb
Test isolation
Each test is independent. The format_string_class fixture provides a fresh instance with reset state.
Future Improvements
- Add integration tests with actual ComfyUI installation
- Add JavaScript tests for frontend functionality
- Increase coverage for circuit_breaker and random_choice nodes
- Add property-based testing with Hypothesis