From 0e0373d0b7809bfbe199cff0df35ef51b837ce2e Mon Sep 17 00:00:00 2001 From: ExponentialML <59846140+ExponentialML@users.noreply.github.com> Date: Thu, 21 Mar 2024 20:22:09 -0700 Subject: [PATCH] Update README.md --- README.md | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 72d0395..3804b51 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,12 @@ # ComfyUI_VisualStylePrompting ComfyUI Version of "Visual Style Prompting with Swapping Self-Attention" +![image](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting/assets/59846140/586304a7-cf61-4f22-8e7d-8bcacbce50de) + > [!NOTE] -> This is WIP. The WIP flag will be removed after more testing and functionality is added. - -![image](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting/assets/59846140/45a2e03a-791c-4b45-bbf1-4adaae7d837b) -*origami bunny* / *a cute dog, origami style, rendered in unreal engine 5* - -![image](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting/assets/59846140/1d3a6cd0-af25-42a6-a8a0-9793f85563d9) -*ControlNet Depth example.* +> This is WIP. +> +> Major changes were made. Please make sure to update your workflows. An updated workflow can be found in the `workflows` directory. Implements the very basics of [Visual Style Prompting](https://github.com/naver-ai/Visual-Style-Prompting)https://github.com/naver-ai/Visual-Style-Prompting by Naver AI. @@ -18,11 +16,24 @@ Clone the reposlitory into your `custom_nodes` folder, and you'll see the node. This has currently only been tested with 1.5 based models. - `reference_image`: The image you wish to reference, -- `conditioning_prompt`: The original positive prompt that you will transfer the style **to** +- `visual_style_prompt`: You **must** use the *Visual Style Prompt Encode* node. This will combine the positive and reference prompts for you. - `reference_image_prompt`: The prompt that describes the reference. You must invoke this pormpt in the conditioning prompt. - `enabled`: Enables or disables the effect. Note that this node will still be hooked even after disabling unless you remove it. +- `denoise`: Works the same way Img2Img works, but utilized with reference and / or init images (this is experimental). +- `input_blocks`: Focuses attention on the encoder layers. +- `middle_block`: Focuses attention on the middle layers. +- `output_blocks`: Focuses attention on the decoder layers. + +> [!TIP] +> In order to get the best results, you must engineer both the positive and reference image prompts correctly. Focus on the details you want to derive from the image reference, and the details you wish to see in the output. +> +> Based on the hero image above, the positive and prompts were as follows: +> +> `kitten art, lines and colors, 3d art, unreal engine 5 render, colors, deep colors, shading, canon 60d`, `lines and colors`. +> +> For this generation, I only wanted the lines and colors of the reference. ## Notes - Currently, this method utilized the VAE Encode & Inpaint method as it needs to iteralively denoise on each step. -Due to how this method works, you'll always get two outputs (the original next to the styled. +Due to how this method works, you'll always get two outputs. To remove the reference latent from the output, simple use a Batch Index Select node.