iGavroche-rocm-ninodes/PHASE2_STARTING_PROMPT.md

ROCM Ninodes Optimization - Phase 2 & 3 Starting Prompt

🎯 Current Status: Phase 1 Complete - Ready for Phase 2

✅ Phase 1 Achievements (COMPLETED)

• 29.1% Performance Improvement achieved (target: 25-35%)
• Memory pooling and adaptive tiling implemented
• Comprehensive instrumentation system deployed
• Baseline performance analysis completed
• Optimization branch created and pushed to GitHub

📊 Current Performance Metrics

• Original Average: 0.001792s
• Phase 1 Optimized: 0.001271s
• Improvement: 29.1% ✅ EXCEEDED TARGET

---

🚀 Next Mission: Phase 2 & 3 Optimizations

Phase 2: Precision and Batch Optimization (Target: 20-30% additional improvement)

Priority: HIGH | Estimated Time: 2-3 hours

Key Focus Areas:

1. Mixed Precision Strategies

   • Implement fp16 accumulation where beneficial for gfx1151
   • Optimize autocast usage patterns
   • Reduce precision overhead
   • Smart precision mode selection

2. Batch Processing Optimization

   • Implement optimal batch sizes for AMD GPUs
   • Optimize memory bandwidth utilization
   • Batch-aware memory management
   • Parallel processing pattern optimization

3. Advanced Memory Management

   • Implement memory prefetching
   • Optimize tensor memory layout
   • Advanced caching strategies
   • Memory fragmentation reduction

Phase 3: Video and Advanced Optimizations (Target: 30-40% additional improvement)

Priority: MEDIUM | Estimated Time: 3-4 hours

Key Focus Areas:

1. Video Processing Optimization

   • Optimal video chunk sizes for temporal consistency
   • Temporal overlap handling optimization
   • Video-specific memory patterns
   • Frame-to-frame processing optimization

2. Advanced Performance Features

   • Performance monitoring integration
   • Adaptive optimization based on usage patterns
   • Advanced memory patterns
   • Real-time optimization adjustments

---

🛠️ Available Tools & Infrastructure

Existing Codebase:

• nodes_v2_phase1.py - Phase 1 optimized VAE decode node
• benchmark_phase1.py - Comprehensive benchmarking system
• instrumentation.py - Performance monitoring system
• test_data/optimization/ - Performance data and results

Development Environment:

• Hardware: GMTek Evo-X2 Strix Halo with gfx1151 GPU, 128GB Unified RAM
• Software: Manjaro Linux with ROCm 6.4+
• Architecture: AMD gfx1151-specific optimizations
• Branch: optimization (ready for Phase 2 work)

Performance Targets:

• Phase 2 Goal: Additional 20-30% improvement (total: ~50-60%)
• Phase 3 Goal: Additional 30-40% improvement (total: ~70-80%)
• Final Target: 50-70% overall improvement

---

📋 Phase 2 Implementation Plan

Step 1: Precision Optimization (1 hour)

# Focus areas:
- Implement smart fp16/fp32 mixed precision
- Optimize autocast usage for gfx1151
- Reduce precision conversion overhead
- Implement precision-aware memory management

Step 2: Batch Processing (1 hour)

# Focus areas:
- Implement optimal batch sizes for AMD GPUs
- Optimize memory bandwidth utilization
- Batch-aware memory management
- Parallel processing optimization

Step 3: Advanced Memory (1 hour)

# Focus areas:
- Memory prefetching implementation
- Advanced tensor memory layout optimization
- Enhanced caching strategies
- Memory fragmentation reduction

Step 4: Benchmarking & Validation (30 minutes)

# Validation:
- Compare Phase 1 vs Phase 2 performance
- Validate 20-30% additional improvement
- Document results and optimizations
- Prepare for Phase 3

---

🎯 Success Criteria for Phase 2

Performance Targets:

• ✅ Additional 20-30% improvement over Phase 1
• ✅ Total improvement: 50-60% over baseline
• ✅ Maintained output quality and compatibility
• ✅ Improved memory efficiency

Technical Requirements:

• ✅ Mixed precision implementation
• ✅ Batch processing optimization
• ✅ Advanced memory management
• ✅ Comprehensive benchmarking validation

---

🔄 Workflow for Phase 2

1. Start Phase 2 Development

# Ensure you're on the optimization branch
git checkout optimization

# Create Phase 2 node implementation
# Build on nodes_v2_phase1.py

2. Implement Optimizations

• Create nodes_v2_phase2.py with precision and batch optimizations
• Implement mixed precision strategies
• Add batch processing optimizations
• Enhance memory management

3. Benchmark and Validate

• Run comprehensive benchmarks
• Compare Phase 1 vs Phase 2 performance
• Validate 20-30% additional improvement
• Document results

4. Prepare for Phase 3

• Commit Phase 2 work
• Update optimization plan
• Prepare Phase 3 implementation

---

📚 Context & Resources

Architecture-Specific Optimizations:

• gfx1151: fp32 preferred, tile sizes 768-1024, conservative memory
• ROCm 6.4+: Disabled TF32, enabled fp16 accumulation, Flash attention
• Memory: Conservative batching, reduced memory modifier

Existing Performance Data:

• Baseline metrics in test_data/optimization/baseline_metrics.json
• Phase 1 results in test_data/optimization/phase1_benchmark_results.json
• Optimization plan in test_data/optimization/optimization_plan.json

Testing Infrastructure:

• Comprehensive mock VAE system
• Performance benchmarking framework
• Instrumentation and monitoring
• Unit and integration tests

---

🎉 Ready to Begin Phase 2

Current Status: ✅ Phase 1 Complete (29.1% improvement) Next Action: 🚀 Implement Phase 2 Optimizations Target: Additional 20-30% improvement (total: 50-60%) Timeline: 2-3 hours for Phase 2 completion

The optimization branch is ready for Phase 2 development. All infrastructure, tools, and baseline data are in place. Let's achieve the next 20-30% improvement!

---

Use this prompt as your starting point for Phase 2 optimization work. All necessary context, tools, and infrastructure are available on the optimization branch.