diff --git a/README.md b/README.md index e8e127a..3f3e294 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ Examples shown here will also often make use of these helpful sets of nodes: - Mixable Motion LoRAs from [original AnimateDiff repository](https://github.com/guoyww/animatediff/) implemented. Caveat: only really work on v2-based motion models like ```mm_sd_v15_v2```, ```mm-p_0.5.pth```, and ```mm-p_0.75.pth``` **(introduced 9/25/23)** - Prompt travel using BatchPromptSchedule node from [ComfyUI_FizzNodes](https://github.com/FizzleDorf/ComfyUI_FizzNodes) **(working since 9/27/23)** - [HotshotXL](https://huggingface.co/hotshotco/Hotshot-XL/tree/main) support (an SDXL motion module arch), ```hsxl_temporal_layers.safetensors``` **(working since 10/05/23)** NOTE: You will need to use ```linear``` beta_schedule, the sweetspot for context_length or total frames (when not using context) is 8 frames, and you will need to use an SDXL checkpoint. Will add more documentation and example workflows soon when I have some time between working on features/other nodes. +- Motion scaling and other motion model settings to influence motion amount **(introduced 10/30/23)** # Upcoming features: - Alternate context schedulers and context types (in progress) @@ -47,7 +48,8 @@ Examples shown here will also often make use of these helpful sets of nodes: # Core Nodes: ## AnimateDiff Loader -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/232ef170-30e0-4119-ace2-4cc9a842d1ac) +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/7dba5a1c-8c29-4d8c-9778-17e217af81b9) + The ***only required node to use AnimateDiff***, the Loader outputs a model that will perform AnimateDiff functionality when passed into a sampling node. @@ -56,8 +58,10 @@ Inputs: - model: model to setup for AnimateDiff usage. ***Must be a SD1.5-derived model.*** - context_options: optional context window to use while sampling; if passed in, total animation length has no limit. If not passed in, animation length will be limited to either 24 or 32 frames, depending on motion model. - motion_lora: optional motion LoRA input; if passed in, can influence movement. +- motion_model_settings: optional settings to influence motion model. - model_name: motion model to use with AnimateDiff. - beta_schedule: noise scheduler for SD. ```sqrt_linear``` is the intended way to use AnimateDiff, with expected saturation. However, ```linear``` can give useful results as well, so feel free to experiment. +- motion_scale: change motion amount generated by motion model - if less than 1, less motion; if greater than 1, more motion. Outputs: - MODEL: model injected to perform AnimateDiff functions @@ -69,14 +73,16 @@ The desired animation length is determined by the latents passed into the sample You can also connect AnimateDiff LoRA Loader nodes to influence the overall movement in the image - currently, only works well on motion v2-based models. -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/524ba030-97aa-47a5-a0fd-ecffbbf5e439) -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/908d1848-13d4-4e86-bdb0-f6870fd28b06) +[Simplest Usage] +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/95c5edc1-3d48-4f03-9f04-4d4ce54c8602) +[All Possible Connections Usage] +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/22ffaf8a-bda9-428d-be11-163cb75feae4) ## Uniform Context Options TODO: fill this out -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/426a2f07-fdcd-477c-bd87-48cf17d91cd0) +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/0327521c-15df-46d8-bee4-6c80b2d7d02d) @@ -93,11 +99,25 @@ Inputs: Outputs: - MOTION_LORA: motion_lora object storing the names of all the LoRAs that were chained behind it - can be plugged into the back of another AnimateDiff LoRA Loader, or into AniamateDiff Loader's motion_lora input. -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/5e46261d-fe87-4daa-8ac3-3ef615f4619d) -![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/21ec6fa6-4874-4312-bd41-477307c9ebf8) +[Simplest Usage] +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/520fb048-1d36-4ec0-a072-532817eafdc0) +[Chaining Multiple Motion LoRAs] +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/9c68fcb1-4573-4ce7-a9a5-4d2b1163d762) +## Motion Model Settings +![image](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/assets/7365912/12887803-8a4f-443b-9c73-82d4b8fc7c75) + +Additional tweaks to the internals of the motion models. The Advanced settings will take a whole guide to explain, and I currently do not have the time for that. Instead, I'll focus on the simple settings. + +Inputs: +- motion_pe_stretch: used to decrease the amount of motion by stretching (and interpolating) between the positional encoders (PEs). TL;DR: number go up, animation slow down. Number up too much, animation begins to vibrate (vibration artifacts). + +Outputs: +- MOTION_MODEL_SETTINGS: motion_model_settings object to be plugged into an AnimateDiff Loader. + + ## Samples (download or drag images of the workflows into ComfyUI to instantly load the corresponding workflows!) ### txt2img