Gavroche
1466b7f75c
- Restored v1.0.13 with all 9 working nodes (ROCMOptimizedCheckpointLoader, ROCMOptimizedKSampler, etc.) - Implemented comprehensive debug flag system (ROCM_NINODES_DEBUG=1) - Created complete test suite with performance and correctness tests - Added data capture infrastructure for real workflow testing - Built standalone test runner with mock data support - Created ComfyUI integration tests for Flux and WAN workflows - Added comprehensive documentation (ARCHITECTURE.md, RULES.md, TESTING_GUIDE.md) - Updated README with testing section and performance targets - All tests passing with proper error handling and data validation Performance Targets: - Flux: 78% improvement (≤110s total) - WAN: 5.6% improvement (≤93s sampling) - VAE Decode: <10s for any resolution - Checkpoint Load: <30s for Flux models Ready for production use with comprehensive testing infrastructure!
5.2 KiB
5.2 KiB
ROCM Ninodes Architecture Documentation
Hardware Configuration
Primary System
- Platform: GMTek Evo-X2 Strix Halo
- GPU: AMD gfx1151 (RDNA 3.5 architecture)
- Memory: 128GB Unified RAM (shared between CPU and GPU)
- OS: Manjaro Linux (kernel 6.17.1-1-MANJARO)
- Shell: Zsh (/usr/bin/zsh)
Software Stack
Core Dependencies
- ROCm: 7.10.0a (latest development build)
- PyTorch: 2.10.0a0+rocm7.10.0a20251011 (nightly build)
- Python: 3.11+ (ComfyUI requirement)
- ComfyUI: Latest main branch with custom optimizations
Environment Configuration
ComfyUI Launch Flags
--use-pytorch-cross-attention # Enable PyTorch cross-attention for better performance
--highvram # Optimize for high VRAM systems
--cache-none # Disable caching to prevent memory issues
Environment Variables
# ROCm-specific optimizations
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 # Enable experimental AOTRITON features
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True # Enable expandable memory segments
# Optional debugging
ROCM_NINODES_DEBUG=0 # Set to 1 to enable data capture for testing
GPU Architecture Specifics
gfx1151 Optimizations
- Precision: fp32 preferred over fp16 for stability
- Tile Sizes: 768-1024 optimal for this architecture
- TF32: Disabled (not supported/beneficial on gfx1151)
- Memory Modifier: Conservative 1.5x for attention operations
- Batch Processing: Conservative approach to prevent OOM
Memory Management Strategy
- Attention Memory: 1.5x modifier for conservative allocation
- VAE Processing: Tiled decoding for large images/videos
- Video Support: 5D tensor processing with 4D output conversion
- Cache Management: Regular
torch.cuda.empty_cache()calls
Library Support Status
HIPBlas
- Status: Fully supported
- Configuration: Automatic detection and optimization
- Performance: Significant speedup for matrix operations
FlashInfer
- Status: Experimental support for gfx1151
- Usage: Enabled in ROCm builds but may have limitations
- Fallback: Standard attention mechanisms when FlashInfer fails
GPU Direct Storage
- Status: Limited support on current setup
- Workaround: Standard file I/O with memory mapping where possible
- Future: May improve with driver updates
Performance Characteristics
Flux Workflow (1024x1024)
- Target Improvement: 78% over baseline
- Checkpoint Load: <30s target
- Sampling: Optimized for gfx1151 architecture
- VAE Decode: <10s target
WAN Workflow (320x320, 17 frames)
- Target Improvement: 5.6% over baseline
- Video Processing: 5D tensor support with 4D output
- Memory Usage: Conservative allocation to prevent OOM
- Total Time: <100s target
Node Architecture
Core Nodes
- ROCMOptimizedCheckpointLoader: Flux-optimized checkpoint loading
- ROCMOptimizedKSampler: Basic sampling with ROCm tuning
- ROCMOptimizedKSamplerAdvanced: Advanced sampling with step control
- ROCMOptimizedVAEDecode: VAE decode with video support
- ROCMOptimizedVAEDecodeTiled: Tiled VAE decode for large images
- ROCMVAEPerformanceMonitor: VAE performance metrics
- ROCMSamplerPerformanceMonitor: Sampler performance metrics
- ROCMFluxBenchmark: Flux workflow benchmarking
- ROCMMemoryOptimizer: Memory management helper
Data Flow
Input → Checkpoint Loader → KSampler → VAE Decode → Output
↓ ↓ ↓ ↓
Monitor Performance Performance Monitor
Testing Infrastructure
Debug Mode
- Environment Variable:
ROCM_NINODES_DEBUG=1 - Data Capture: Saves input/output tensors to pickle files
- Performance Impact: Zero when disabled
- Location:
test_data/captured/
Test Categories
- Unit Tests: Individual node functionality
- Performance Tests: Timing and memory usage validation
- Integration Tests: Full workflow execution
- Regression Tests: Prevent performance degradation
Constraints and Limitations
Hardware Constraints
- Memory: 128GB shared RAM limits batch sizes
- GPU: gfx1151 specific optimizations required
- Storage: No GPU Direct Storage support
Software Constraints
- ROCm Version: Requires 7.10.0a+ for full features
- PyTorch: Nightly builds required for latest optimizations
- ComfyUI: Custom modifications needed for optimal performance
Performance Constraints
- Precision: fp32 preferred over fp16 for stability
- Memory: Conservative allocation to prevent OOM
- Batch Size: Limited by available memory
- Video Processing: 5D tensor handling complexity
Future Considerations
Planned Improvements
- FlashInfer: Full support when available
- Memory Optimization: Better memory management strategies
- Batch Processing: Larger batch sizes with memory improvements
- Video Support: Enhanced video processing capabilities
Research Areas
- Attention Mechanisms: Optimize for gfx1151
- Memory Layout: Better tensor organization
- Pipeline Optimization: Reduce data movement overhead
- Quantization: Explore fp16 where stable