fix: don't group LoRAs with zero overlap in Compatibility Analyzer
The +0.1 compatibility bonus was applied to all non-opposing pairs, including LoRAs that share no weights (like DPO LoRAs touching different layers). This caused them to be grouped instead of solo. Now the bonus only applies when overlap > 0, so LoRAs with no shared weights correctly end up as solo. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
a3c40dfac8
commit
284bdbc954
+4
-1
@@ -8113,11 +8113,14 @@ class LoRACompatibilityAnalyzer(LoRAOptimizer):
|
||||
@staticmethod
|
||||
def _compute_compat_matrix(pairwise_similarities, pairwise_conflicts, n_loras):
|
||||
conflict_by_pair = {(pc["i"], pc["j"]): pc["ratio"] for pc in pairwise_conflicts}
|
||||
overlap_by_pair = {(pc["i"], pc["j"]): pc["overlap"] for pc in pairwise_conflicts}
|
||||
matrix = [[0.0] * n_loras for _ in range(n_loras)]
|
||||
for (i, j), cos_sim in pairwise_similarities.items():
|
||||
conflict_ratio = conflict_by_pair.get((i, j), 0.0)
|
||||
overlap = overlap_by_pair.get((i, j), 0)
|
||||
compat = cos_sim * (1.0 - conflict_ratio)
|
||||
if cos_sim > -0.1:
|
||||
# Only boost non-opposing pairs that actually share weights
|
||||
if cos_sim > -0.1 and overlap > 0:
|
||||
compat += 0.1
|
||||
matrix[i][j] = compat
|
||||
matrix[j][i] = compat
|
||||
|
||||
Reference in New Issue
Block a user