Update nodes.py

streamline simple node settings

README.md

@@ -36,6 +36,13 @@ Still, its far from perfect. With "medium" setting we get an image that is much
 **Reflux medium strength**
 ![image](https://github.com/user-attachments/assets/b632457a-3a7e-4d99-981e-6c2682d16e2e)
 
+
+You can also mix more than one images together. Here is an example with adding a second image: [[https://www.pexels.com/de-de/foto/komplizierte-bogen-der-mogul-architektur-in-jaipur-29406307/]]
+
+Mixing both together and using the anime prompt above gives me
+
+![image](https://github.com/user-attachments/assets/1385b22f-4497-4fdf-8255-3a15bda74a1d)
+
 Finally, we try a very challenging prompt: "Marble statues, sculptures, stone statues. stone and marble texture. Two sculptures made out of marble stone.". As you can see, I repeated the prompt multiple times to increase its strength.
 But despite the repeats, the default Reflux workflow will just give us the input image refluxed - our prompt is totally ignored.
 
@@ -53,6 +60,14 @@ Further decreasing the Reflux strength will transform the woman into statues fin
 
 You can use the images above for example workflows.
 
+Since last update I added a new node which I would recommend over the old workflow above. Use this image as an example workflow:
+
+![image](https://github.com/user-attachments/assets/b6ee8e4e-2599-499d-9dd4-7fbdc5879e90)
+
+There are two parameters you can play around with:
+- downsampling_factor: as larger the value as less information you get from your image. Use a value between 1 and 4 with 1 is the original reflux method. For all example images I used a downsampling_factor of 3, which usually works best. The image above, however, uses a downsampling_factor of 2 and, as you can see, is even closer to the original image while still having a clear anime style
+- mode: the method used for downsampling. "area" or "bicubic" work best.
+
 The ComfyUI plugin comes with two additional nodes: StyleModelApplySimple and StyleModelApplyAdvanced. Usually, you can just replace your ApplyStyle node with the StyleModelApplySimple node and "medium" strength and you will get best results. However, feel free to experiment with the StyleModelApplyAdvanced node.
 
 ## Short background on Reflux
@@ -67,10 +82,16 @@ So why is Reflux dominating the final prompt? It's because the user prompt is us
 
 So there are two solutions here: Either we shrink the strength of the Reflux prompt, or we shorten the Reflux prompt.
 
-## Token merging by downsampling
-To shrink the Reflux prompt and increase the influence of the user prompt, we can use a simple trick: We take the 27x27 image patches and split them into 9x9 blocks, each containing 3x3 patches. We then merge all 3x3 tokens into one by averaging its latent embedding. So instead of having a very long prompt with 27x27=729 tokens we now only have 9x9=81 tokens. So our newly added prompt is much smaller than the user provided prompt and, thus, have less influence on the image generation.
+The next sections are a bit chaotic: I changed the method several times and many stuff I tried is outdated already. The only and best technique I found so far is described in **Interpolation methods**.
 
-Downsampling is what happens when you use the "medium" setting. Of all three techniques I tried to decrease the Reflux effect, downsampling worked best. However, there are no further customization options. You can only downsample to 81 tokens (downsampling more is too much).
+## Controling Reflux with Token downsampling
+To shrink the Reflux prompt and increase the influence of the user prompt, we can use a simple trick: We take the 27x27 image patches and split them into 9x9 blocks, each containing 3x3 patches. We then merge all 3x3 tokens into one by averaging their latent embeddings. So instead of having a very long prompt with 27x27=729 tokens we now only have 9x9=81 tokens. So our newly added prompt is much smaller than the user provided prompt and, thus, have less influence on the image generation.
+
+Downsampling is what happens when you use the "medium" setting. Of all three techniques I tried to decrease the Reflux effect, downsampling worked best. ~~However, there are no further customization options. You can only downsample to 81 tokens (downsampling more is too much)~~.
+
+## Interpolation methods
+
+Instead of averaging over small blocks of tokens, we can use a convolution function to shrink our 27x27 images patches to an arbitrary size. There are different functions available which most of you probably know from image resizing (its the same procedure). The averaging method above is "area", but there are also other methods available such as "bicubic".  
 
 ## Controling Reflux with Token merging
 

nodes.py

@@ -32,10 +32,8 @@ def automerge(tensor, threshold):
         newTensor.append(torch.stack(tokens))
     return torch.stack(newTensor)
 
-DownScalingFactors = {"1:1": 1, "1:3":3, "1:9": 9}
-STRENGTHS = ["very high", "medium", "low", "very low", "lowest"]
-STRENGTHS_VALUES = [1,3,3,3,3]
-WEIGHTING_VALUES = [1,1,0.75, 0.35,0.2]
+STRENGTHS = ["highest", "high", "medium", "low", "lowest"]
+STRENGTHS_VALUES = [1,2,3,4,5]
 
 class StyleModelApplySimple:
     @classmethod
@@ -51,15 +49,15 @@ class StyleModelApplySimple:
     CATEGORY = "conditioning/style_model"
 
     def apply_stylemodel(self, clip_vision_output, style_model, conditioning, image_strength):
-        cond = style_model.get_cond(clip_vision_output).flatten(start_dim=0, end_dim=1).unsqueeze(dim=0)
         stren = STRENGTHS.index(image_strength)
-        factor = STRENGTHS_VALUES[stren]
-        weighting = WEIGHTING_VALUES[stren]
-        if factor>1:
+        downsampling_factor = STRENGTHS_VALUES[stren]
+        mode="area" if downsampling_factor==3 else "bicubic"
+        cond = style_model.get_cond(clip_vision_output).flatten(start_dim=0, end_dim=1).unsqueeze(dim=0)
+        if downsampling_factor>1:
             (b,t,h)=cond.shape
             m = int(np.sqrt(t))
-            cond=torch.nn.PixelUnshuffle(factor)(cond.view(b, m, m, h).transpose(-1, 1)).transpose(1,-1)
-            cond=cond.view(b,m//factor, m//factor, h, factor*factor).mean(-1).view(b,-1, h) * weighting
+            cond=torch.nn.functional.interpolate(cond.view(b, m, m, h).transpose(1,-1), size=(m//downsampling_factor, m//downsampling_factor), mode=mode)#
+            cond=cond.transpose(1,-1).reshape(b,-1,h)
         c = []
         for t in conditioning:
             n = [torch.cat((t[0], cond), dim=1), t[1].copy()]
@@ -101,23 +99,23 @@ class StyleModelApplyAdvanced:
         return {"required": {"conditioning": ("CONDITIONING", ),
                              "style_model": ("STYLE_MODEL", ),
                              "clip_vision_output": ("CLIP_VISION_OUTPUT", ),
-                             "downscale": (["1:1", "1:3", "1:9"], {"default": "1:3"}),
+                             "downsampling_factor": ("INT", {"default": 3, "min": 1, "max": 9}),
+                             "mode": (["nearest", "bicubic", "bilinear","area", "nearest-exact" ], {"default": "area"}),
                              "merge_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
                              "clipWeight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01})
                              }}
     RETURN_TYPES = ("CONDITIONING",)
     FUNCTION = "apply_stylemodel"
 
-    CATEGORY = "conditioning/style_model"
+    CATEGORY = "_for_testing/conditioning/style_model"
 
-    def apply_stylemodel(self, clip_vision_output, style_model, conditioning, downscale, merge_strength, clipWeight):
+    def apply_stylemodel(self, clip_vision_output, style_model, conditioning, downsampling_factor, mode, merge_strength, clipWeight):
         cond = style_model.get_cond(clip_vision_output).flatten(start_dim=0, end_dim=1).unsqueeze(dim=0)
-        if downscale!="1:1":
+        if downsampling_factor>1:
             (b,t,h)=cond.shape
             m = int(np.sqrt(t))
-            factor = DownScalingFactors[downscale]
-            cond=torch.nn.PixelUnshuffle(factor)(cond.view(b, m, m, h).transpose(-1, 1)).transpose(1,-1)
-            cond=cond.view(b,m//factor, m//factor, h, factor*factor).mean(-1).view(b,-1, h)
+            cond=torch.nn.functional.interpolate(cond.view(b, m, m, h).transpose(1,-1), size=(m//downsampling_factor, m//downsampling_factor), mode=mode)#
+            cond=cond.transpose(1,-1).reshape(b,-1,h)
 
         cond = automerge(cond, merge_strength)*clipWeight
         c = []
@@ -139,7 +137,7 @@ NODE_CLASS_MAPPINGS = {
 
 # A dictionary that contains the friendly/humanly readable titles for the nodes
 NODE_DISPLAY_NAME_MAPPINGS = {
-    "StyleModelApplySimple": "Apply Style model (simple)",
+    "StyleModelApplySimple": "Apply style model (simple)",
     "StyleModelApplyAdvanced": "Apply Style model (advanced)",
-    "StyleModelApplyInterpolation": "Apply style model (interpolation)"
-}
+    "StyleModelApplyInterpolation": "Apply style model (advanced)"
+}