improve and correct merger operations based on papers

docs/merger_2_model_modes.md

@@ -23,12 +23,47 @@
 ---
 
 ## Power-Up (DARE)
-> Adds the unique capabilities of Model B to Model A using the Drop and Rescale (DARE) technique, which often preserves the knowledge of the base model better than simple additions.
+> Adds the unique capabilities of Model B to Model A using the Drop and Rescale (DARE) technique. This implementation handles shape mismatches between models by padding and uses a randomized dropout mask.
 
 **Models Used:** A, B
 **Parameters:**
-- **Alpha:** The dropout rate. This is the proportion of unique weights from Model B that are *dropped* before merging. A higher value means less of B is merged.
+- **Alpha:** The dropout rate ($p$). This is the proportion of delta parameters from Model B that are randomly set to zero.
 - **Beta:** A final multiplier for the rescaled difference before it's added to Model A.
+- **Rescaling Logic:** Remaining weights are automatically rescaled by $1/(1-p)$ as per the DARE paper to approximate the original embeddings.
+
+---
+
+## Enhanced Man Interp
+> Sophisticated interpolation between values from A and B depending on their difference relative to other values, with manual threshold control.
+
+**Models Used:** A, B
+**Parameters:**
+- **Alpha:** Interpolation strength.
+- **Beta:** Lower mean threshold for filtering differences.
+- **Gamma:** Upper mean threshold for filtering differences.
+- **Delta:** Smoothness factor (mix between randomized mask and powered differences).
+
+---
+
+## Enhanced Auto Interp
+> Automated version of the enhanced interpolation mode that dynamically calculates thresholds based on mean differences.
+
+**Models Used:** A, B
+**Parameters:**
+- **Alpha:** Interpolation strength.
+- **Beta:** Threshold adjustment factor.
+- **Gamma:** Smoothness factor.
+
+---
+
+## Weight-Sum Cutoff
+> A linear interpolation mode that only applies the merge to weights whose differences fall within a specific threshold range.
+
+**Models Used:** A, B
+**Parameters:**
+- **Alpha:** Interpolation weight (multiplier for the difference).
+- **Beta:** Upper threshold for the difference cutoff.
+- **Gamma:** Lower threshold for the difference cutoff.
 
 ---
 
@@ -84,4 +119,4 @@ Layers matching any pattern will be **removed entirely** from the output.
 | `text_model` | All text encoder layers |
 | `\.norm` | All normalization layers |
 | `attn\.(q\|k\|v)` | Query, key, value attention weights |
-| `block\.[0-9]\.` | Blocks 0-9 |
+| `block\.[0-9]\.` | Blocks 0-9 |

docs/merger_3_model_modes.md

@@ -22,19 +22,19 @@
 ---
 
 ## Extract-Features
-> A powerful mode that identifies features present in both `(B - A)` and `(C - A)` and adds them to A. Allows for fine-grained control over combining aspects based on their similarity.
+> A powerful mode that identifies features present in both `(B - A)` and `(C - A)` and adds them to A. It uses per-vector cosine similarity to decide how much of each feature to keep, allowing for fine-grained control over combining aspects.
 
 **Models Used:** A, B, C
 **Parameters:**
 - **Alpha:** Weights the merge between Model B (`0.0`) and Model C (`1.0`).
 - **Beta:** Controls the focus on similarity (`0.0`) versus dissimilarity (`1.0`).
-- **Gamma:** A bias exponent for similarity. Higher values increase the bias.
+- **Gamma:** A bias exponent for similarity calculation.
 - **Delta:** A final multiplier for the extracted features before they are added to Model A.
 
 ---
 
 ## Add-Dissimilarities
-> Identifies features that are dissimilar between Model B and Model C and adds them to Model A. Useful for combining unique aspects of two different models.
+> Identifies features that are dissimilar between Model B and Model C (relative to A) and adds them to Model A. Useful for combining unique aspects of two different models.
 
 **Models Used:** A, B, C
 **Parameters:**
@@ -80,4 +80,4 @@ Layers matching any pattern will be **removed entirely** from the output.
 **Pattern format:**
 - Whitespace-separated regex patterns
 - Patterns use **substring matching** (not full match)
-- Example: `text_model lora` matches any key containing "text_model" OR "lora"
+- Example: `text_model lora` matches any key containing "text_model" OR "lora"