--- tags: - AnimateDiff - Animation --- # 🧪Inject I2V into AnimateDiff Model 🎭🅐🅓② ## Documentation - Class name: `ADE_InjectI2VIntoAnimateDiffModel` - Category: `Animate Diff 🎭🅐🅓/② Gen2 nodes ②/AnimateLCM-I2V/🧪experimental` - Output node: `False` This node is designed to integrate the I2V (Image to Video) model into the AnimateDiff framework, enhancing its capabilities by incorporating motion models. It serves as a bridge to enrich AnimateDiff's animation process with additional motion dynamics, facilitating a more complex and nuanced animation output. ## Input types ### Required - **`model_name`** - Specifies the name of the motion model to be loaded, playing a crucial role in determining the animation's motion dynamics. - Comfy dtype: `COMBO[STRING]` - Python dtype: `str` - **`motion_model`** - Represents the motion model object to be injected into the AnimateDiff model, crucial for applying specific motion dynamics to the animation process. - Comfy dtype: `MOTION_MODEL_ADE` - Python dtype: `MotionModelPatcher` ### Optional - **`ad_settings`** - Optional settings for the AnimateDiff process, allowing for customization of the animation's appearance and behavior. - Comfy dtype: `AD_SETTINGS` - Python dtype: `AnimateDiffSettings` ## Output types - **`MOTION_MODEL`** - Comfy dtype: `MOTION_MODEL_ADE` - The enhanced AnimateDiff model with the injected I2V capabilities, ready for animation tasks. - Python dtype: `MotionModelPatcher` ## Usage tips - Infra type: `CPU` - Common nodes: unknown ## Source code ```python class LoadAnimateDiffAndInjectI2VNode: @classmethod def INPUT_TYPES(s): return { "required": { "model_name": (get_available_motion_models(),), "motion_model": ("MOTION_MODEL_ADE",), }, "optional": { "ad_settings": ("AD_SETTINGS",), } } RETURN_TYPES = ("MOTION_MODEL_ADE",) RETURN_NAMES = ("MOTION_MODEL",) CATEGORY = "Animate Diff 🎭🅐🅓/② Gen2 nodes ②/AnimateLCM-I2V/🧪experimental" FUNCTION = "load_motion_model" def load_motion_model(self, model_name: str, motion_model: MotionModelPatcher, ad_settings: AnimateDiffSettings=None): # make sure model w/ encoder actually has encoder if motion_model.model.img_encoder is None: raise Exception("Passed-in motion model was expected to have an img_encoder, but did not.") # load motion module and motion settings, if included loaded_motion_model = load_motion_module_gen2(model_name=model_name, motion_model_settings=ad_settings) inject_img_encoder_into_model(motion_model=loaded_motion_model, w_encoder=motion_model) return (loaded_motion_model,) ```