Three compounding bugs caused saved LoRAs to be 2.3GB (instead of ~350MB) and
produce absurd images:
1. QKV refusion expanded all SVD-compressed LoRAAdapters to full-rank float32
diffs, destroying all compression. SaveMergedLoRA saw 0 low-rank patches.
2. With no surviving LoRAAdapters, auto-rank fell back to 128 — too low for
a merge of two rank-128 LoRAs (needs sum_rank=256), losing ~50% of signal.
3. _compress_to_lowrank returned float32 tensors (SVD needs float32 internally
but never cast back), doubling file size vs expected fp16.
Fixes:
- _compress_to_lowrank now preserves input dtype (also halves merge-time RAM)
- QKV refusion downcasts fused diffs back to original dtype
- Optimizer passes sum_rank in lora_data so SaveMergedLoRA uses correct rank
- SaveMergedLoRA detects native dtype and enforces it on all saved tensors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
