From 6b4e6fea75ebff0c13c3c501946ce2a05426c5ce Mon Sep 17 00:00:00 2001 From: kaibioinfo Date: Fri, 22 Nov 2024 20:47:44 +0100 Subject: [PATCH] Update README.md spelling errors --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index fe8db89..bd5a3cd 100644 --- a/README.md +++ b/README.md @@ -67,8 +67,8 @@ So why is Reflux dominating the final prompt? It's because the user prompt is us So there are two solutions here: Either we shrink the strength of the Reflux prompt, or we shorten the Reflux prompt. -## Token merging by downsampling -To shrink the Reflux prompt and increase the influence of the user prompt, we can use a simple trick: We take the 27x27 image patches and split them into 9x9 blocks, each containing 3x3 patches. We then merge all 3x3 tokens into one by averaging its latent embedding. So instead of having a very long prompt with 27x27=729 tokens we now only have 9x9=81 tokens. So our newly added prompt is much smaller than the user provided prompt and, thus, have less influence on the image generation. +## Controling Reflux with Token downsampling +To shrink the Reflux prompt and increase the influence of the user prompt, we can use a simple trick: We take the 27x27 image patches and split them into 9x9 blocks, each containing 3x3 patches. We then merge all 3x3 tokens into one by averaging their latent embeddings. So instead of having a very long prompt with 27x27=729 tokens we now only have 9x9=81 tokens. So our newly added prompt is much smaller than the user provided prompt and, thus, have less influence on the image generation. Downsampling is what happens when you use the "medium" setting. Of all three techniques I tried to decrease the Reflux effect, downsampling worked best. However, there are no further customization options. You can only downsample to 81 tokens (downsampling more is too much).