smoke_test.py - Basic smoke tests to verify imports, structure, and core functionality
test_simple_real.py - Comprehensive tests for agent improvements with mocked responses
test_kluster_real.py - Real API integration tests using Kluster.ai endpoint
test_hash_utils.py - Tests for utility functions and hash handling
test_agent_structure.py - Tests for code structure, documentation, and naming improvements

🚀 LightRAG Integration Tests

test_lightrag_integration.py - Core LightRAG + LiteLLM compatibility tests using Kluster API
test_lightrag_incremental_documents.py - Advanced multi-document processing and cross-document querying tests

Configuration Files

conftest.py - pytest configuration with dependency mocking and fixtures
__init__.py - Makes tests directory a Python package

Running Tests

Run All Tests

python3 -m pytest tests/ -v

Run Specific Test Categories

# Basic functionality tests
python3 -m pytest tests/smoke_test.py tests/test_simple_real.py -v

# Real API tests (requires internet)
python3 -m pytest tests/test_kluster_real.py -v

# LightRAG integration tests (requires lightrag-hku)
python3 -m pytest tests/test_lightrag_integration.py -v
python3 -m pytest tests/test_lightrag_incremental_documents.py -v

# Structure and documentation tests
python3 -m pytest tests/test_agent_structure.py -v

Run with Markers

# API tests only
python3 -m pytest -m api -v

# Integration tests only  
python3 -m pytest -m integration -v

Test Categories

🚀 Smoke Tests (`smoke_test.py`)

Import verification
Display name mappings
Basic structure validation
Documentation existence
Error handling patterns

🔧 Simple Real Tests (`test_simple_real.py`)

Direct API connectivity test
Input validation comprehensive testing
Naming improvements verification
Documentation structure validation
Mock functionality testing

🌐 Kluster Real Tests (`test_kluster_real.py`)

Real API calls to Kluster.ai endpoint
Agent node integration with live API
Multi-iteration processing
Multiple prompts handling
Recursion filter integration

🛠️ Hash Utils Tests (`test_hash_utils.py`)

Input hashing for different data types
Agent node input handling
Callable and image processing
Message and list handling

📋 Structure Tests (`test_agent_structure.py`)

File organization verification
Display name improvements
Docstring completeness
Method documentation quality
Code structure enhancements

🧠 LightRAG Integration Tests

Core Integration (`test_lightrag_integration.py`)

Node Availability: Verification that all LightRAG nodes can be instantiated
Wrapper Function Signature: Tests the async compatibility wrapper that bridges LiteLLM and LightRAG
Document Processor Integration: Real integration testing with Kluster API
Compatibility Verification: Ensures LiteLLM completion providers work seamlessly with LightRAG

Advanced Scenarios (`test_lightrag_incremental_documents.py`)

Incremental Document Processing: Multi-document knowledge graph building
Cross-Document Querying: Relationship discovery across multiple processed documents
Agent Memory Integration: Testing memory provider functionality with incremental data
Working Directory Persistence: State management across sessions
Performance Monitoring: API call efficiency and processing time analysis

Key Features Tested

✅ Fixed Issues

Infinite loop bug: Fixed in LiteLLM completion handler
Import errors: Conditional LightRAG imports
Error handling: Proper exception management
Test isolation: Mock dependencies prevent conflicts
LightRAG Compatibility: Async wrapper for LiteLLM synchronous functions

✅ Agent Improvements Verified

Input validation with comprehensive error messages
Better naming conventions and display names
Extensive documentation (USAGE_GUIDE.md, EXAMPLES.md)
Robust error handling and warnings
Progress indicators and user feedback
Caching system functionality

✅ LightRAG Integration Verified

Universal Model Support: Any LiteLLM model works with LightRAG
Async Compatibility: Automatic wrapper converts sync to async functions
Prompt Combination: Intelligent merging of system prompts, history, and user input
State Persistence: Working directories maintain knowledge graphs across sessions
Error Recovery: Robust handling of API failures and edge cases
Performance Optimization: Efficient API usage and call patterns

API Configuration

Tests use the Kluster.ai API endpoint:

Base URL: https://api.kluster.ai/v1
Model: mistralai/Mistral-Nemo-Instruct-2407
API Key: Configured in test fixtures

Test Results Summary

As of the latest updates:

All smoke tests: ✅ PASS
Simple real tests: ✅ PASS
Kluster API tests: ✅ PASS
Hash utils tests: ✅ PASS
Structure tests: ✅ PASS
LightRAG integration tests: ✅ PASS (3/3 core integration tests)
LightRAG incremental tests: ✅ PASS (4/4 advanced scenario tests)

Total: 39+ tests all passing, comprehensive coverage of all features

LightRAG Test Highlights

✅ Real API Integration: Tests verified with actual Kluster API calls
✅ Multi-Document Processing: Incremental knowledge graph building confirmed
✅ Cross-Document Queries: Relationship discovery across documents working
✅ State Persistence: Working directory management confirmed
✅ Universal Compatibility: Any LiteLLM model works with LightRAG

Dependencies

Automatically Mocked Dependencies

ComfyUI modules (folder_paths, execution, server)
Config and utils modules

Real Dependencies for LightRAG Tests

lightrag-hku - Required for LightRAG integration tests
litellm - Universal LLM interface
numpy - Vector operations
networkx - Graph processing

Test-Only Dependencies

pytest - Test framework
Kluster API access - For real integration tests

Note: LightRAG tests automatically skip if dependencies aren't available

Troubleshooting

Import Errors

The test suite automatically handles import issues with dependency mocking in conftest.py.

API Connection Issues

If API tests fail, check:

Internet connectivity
Kluster.ai service availability
API endpoint status

Timeout Issues

Tests have a 30-second timeout to prevent infinite loops. If tests hang:

Check for infinite retry loops in LiteLLM calls
Verify API endpoints are responding
Review error handling in agent code

Contributing

When adding new tests:

Follow the existing naming convention
Add appropriate pytest markers (@pytest.mark.api, etc.)
Include proper documentation
Ensure tests clean up after themselves
Mock external dependencies appropriately

Recent Improvements

Test Suite Cleanup

✅ Removed all mocking from LightRAG tests - now use real functionality
✅ Updated to Kluster API - all tests use production-ready Kluster endpoint
✅ Eliminated redundant files - streamlined to essential, working tests
✅ Fixed infinite loop issues - robust error handling and timeouts
✅ Real integration validation - comprehensive testing with actual API calls

LightRAG Integration Achievement

✅ Production-ready integration between LightRAG and LiteLLM
✅ Universal model support - any LiteLLM model works with LightRAG
✅ Async compatibility wrapper - seamlessly bridges synchronous and async interfaces
✅ Comprehensive test coverage - from basic integration to advanced multi-document scenarios
✅ Performance validation - efficient API usage patterns confirmed

Focus is on production-ready, thoroughly tested functionality that users can rely on in real workflows.

README.md

ComfyUI LiteLLM Test Suite

Test Structure

Core Test Files