nodes/balarooty-comfyui-extend
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
main
balarooty-comfyui-extend/docs/tutorial.md
T

19 KiB
Raw Permalink Blame History

ComfyUI-LTXFlow Tutorial

A step-by-step guide to creating AI video from still images using Qwen Image Edit and LTX Video 2.3, orchestrated by the LTXFlow custom nodes.

Table of Contents

  1. What This Pack Does
  2. Prerequisites
  3. Installation
  4. Downloading Models
  5. Workflow 05 — Qwen Edit to LTX First/Last
  6. Workflow 04 — LTX First/Last (No Qwen)
  7. Node Reference
  8. Building a Multi-Scene Video
  9. Recommended Settings
  10. Troubleshooting

What This Pack Does

ComfyUI-LTXFlow adds workflow glue between Qwen Image Edit 2511 and LTX Video 2.3. The core idea:

Source photo  →  Qwen edits it  →  LTX animates from original to edited

The pack provides seven custom nodes that handle frame extraction, guide injection, image bridging, and scene stitching. It does not replace the official LTX or Qwen nodes — it connects them.

The Pipeline at a Glance

┌─────────────┐      ┌──────────────────┐      ┌───────────────────────┐
│  LoadImage   │─────▶│  Qwen Image Edit │─────▶│  Qwen Edit Bridge     │
│  (source)    │      │  (2511 subgraph) │      │  (resize + prepare)   │
└─────────────┘      └──────────────────┘      └───────┬───────────────┘
                                                        │
                                              first_frame + last_frame
                                                        │
                                                        ▼
                                               ┌─────────────────┐
                                               │  First/Last     │
                                               │  Guide          │
                                               │  (inject into   │
                                               │   LTX cond.)    │
                                               └────────┬────────┘
                                                        │
                                                        ▼
                                               ┌─────────────────┐
                                               │  LTX 2.3        │
                                               │  Sampler +      │
                                               │  Upscale +      │
                                               │  Decode         │
                                               └────────┬────────┘
                                                        │
                                                        ▼
                                               ┌─────────────────┐
                                               │  SaveVideo      │
                                               └─────────────────┘

Prerequisites

  • ComfyUI — latest version (update guide)
  • GPU — 24 GB VRAM recommended (48 GB for best quality). The fp8 models can run on 24 GB cards.
  • Python 3.10+
  • aria2 — the download script installs it automatically if missing

Installation

1. Clone or Copy the Node Pack

Place this repository inside your ComfyUI custom nodes folder:

cd /path/to/ComfyUI/custom_nodes/
git clone https://github.com/balarooty/comfyui-extend.git ComfyUI-LTXFlow

Or if you already have the folder locally, symlink it:

ln -s /path/to/comfyui-extend /path/to/ComfyUI/custom_nodes/ComfyUI-LTXFlow

2. Install Required Custom Node Packs

The workflows depend on three external packs. Install them automatically:

COMFYUI_DIR=/path/to/ComfyUI bash scripts/install_custom_nodes.sh

This installs:

Pack Provides
ComfyUI-KJNodes LazySwitchKJ, VAELoaderKJ, LTX2SamplingPreviewOverride
WhatDreamsCost-ComfyUI LTXSequencer, MultiImageLoader
ComfyUI Easy Use easy mathInt

Alternatively, install through ComfyUI Manager if you prefer the GUI.

3. Restart ComfyUI

After installing, restart the ComfyUI server and refresh the browser.

Downloading Models

The download script fetches every model needed by the main workflow:

COMFYUI_DIR=/path/to/ComfyUI bash scripts/download_models.sh

Set COMFYUI_DIR to your actual ComfyUI root. The script downloads:

Qwen Image Edit 2511 Models

File Directory Size (approx.)
qwen_image_edit_2511_bf16.safetensors models/diffusion_models/ ~14 GB
qwen_2.5_vl_7b_fp8_scaled.safetensors models/text_encoders/ ~8 GB
qwen_image_vae.safetensors models/vae/ ~150 MB
Qwen-Image-Edit-2511-Lightning-4steps-V1.0-bf16.safetensors models/loras/ ~14 GB

LTX Video 2.3 Models

File Directory Size (approx.)
ltx-2.3-22b-dev-fp8.safetensors models/checkpoints/ ~22 GB
gemma_3_12B_it_fp8_scaled.safetensors models/text_encoders/ ~12 GB
ltx-2.3-22b-distilled-lora-384.safetensors models/loras/ ~700 MB
ltx-2.3-spatial-upscaler-x2-1.1.safetensors models/latent_upscale_models/ ~500 MB

Optional: Hugging Face Auth

If any model is gated:

HF_TOKEN=hf_xxxxx COMFYUI_DIR=/path/to/ComfyUI bash scripts/download_models.sh

Optional: Custom Download Speed

The script uses aria2 with 16 parallel connections by default:

ARIA2_CONNECTIONS=8 ARIA2_SPLITS=8 COMFYUI_DIR=/path/to/ComfyUI bash scripts/download_models.sh

Workflow 05 — Qwen Edit to LTX First/Last

This is the main workflow. It takes a source image, runs it through Qwen Image Edit to create a modified version, then animates the transition using LTX Video 2.3.

Loading the Workflow

  1. Open ComfyUI in your browser
  2. Click Load → select workflows/05_qwen_edit_to_ltx_first_last.json
  3. If ComfyUI shows "Installation Required", install the missing packs (see Installation)

Step-by-Step Walkthrough

Step 1: Load Your Source Image

Find the LoadImage node on the left side of the canvas. Upload your source image here. This is the starting frame of your video.

Tip: Use images with clear subjects. Portraits, objects, and scenes with distinct features work best.

Step 2: Load a Reference/Style Image

Find the second LoadImage node. Upload a reference image that shows the target look, material, or style you want Qwen to apply to the source.

For example:

  • Source: a leather sofa
  • Reference: a velvet texture photo
  • Result: Qwen edits the sofa to look velvet, then LTX animates the transformation

Step 3: Set the Prompt

In the Model Loader subgraph (node labeled with a "↓ Select your own models" note), find the Prompt field. Write a description of the final scene.

Example prompts:

  • "A cozy velvet sofa in a warm living room, soft lighting"
  • "A futuristic cityscape at sunset, neon lights reflecting on wet streets"

Step 4: Configure Video Length

In the same Model Loader subgraph, set length (seconds). Start with 5 seconds for testing. Each second generates about 24 frames.

Step 5: Queue the Prompt

Click Queue Prompt (or press Ctrl+Enter). The execution order is:

  1. Qwen Image Edit processes the source + reference into an edited image
  2. The Qwen Edit Bridge prepares first frame (source) and last frame (edited)
  3. First/Last Guide injects both frames into LTX conditioning
  4. LTX 2.3 generates the video with 3-stage upscaling
  5. The decoded video is saved

Understanding the Qwen Edit Bridge

The bridge node (LTX Flow - Qwen Edit Bridge) has two important settings:

Setting Options What It Does
first_frame_source source_image / qwen_edited_image Which image becomes the first frame of the video
resize_to source_image / qwen_edited_image Which image's dimensions are used as the target size

Default behavior: The source image is the first frame, the Qwen-edited image is the last frame. This creates a smooth transformation video.

Alternative: Set first_frame_source = qwen_edited_image to animate only within the edited scene (no visible transition from original).

Understanding First/Last Guide

The guide node (LTX Flow - First/Last Guide) controls how strongly the first and last frames influence generation:

Parameter Default Range Notes
first_strength 1.0 0.0–1.0 How strongly the first frame anchors frame 0
last_strength 1.0 0.0–1.0 How strongly the last frame anchors the final frame
last_frame_index -1 any integer -1 = last frame of the latent. Use a specific number for earlier anchoring.

Tip: Keep both strengths at 1.0 for reliable transitions. Lower the last strength to 0.7–0.8 if you want more creative freedom in how LTX reaches the target frame.

Workflow 04 — LTX First/Last (No Qwen)

If you just want to use LTX Video 2.3 with first/last frame guidance (without Qwen editing), use:

workflows/04_ltxflow_first_last_wdc_ltx23.json

This is based on the WhatDreamsCost reference workflow. The key difference is that the dynamic LTXSequencer section is replaced with our First/Last Guide node.

How to Use

  1. Load the workflow
  2. In the MultiImageLoader node, load your first and last frame images
  3. The Extract Frame nodes automatically pull index 0 (first) and index -1 (last)
  4. Set your prompt and video length in the Model Loader subgraph
  5. Queue the prompt

Node Reference

LTX Flow - Extract Frame

Category: LTXFlow/Frame Tools

Pulls a single frame from an image batch.

Input Type Description
video_frames IMAGE A batch of frames
index INT Frame index. 0 = first, -1 = last, -2 = second-to-last, etc.
Output Type Description
frame IMAGE The extracted single frame
resolved_index INT The actual positive index that was used

LTX Flow - Extract Tail

Category: LTXFlow/Frame Tools

Extracts the last N frames from a clip. Essential for continuation workflows.

Input Type Description
video_frames IMAGE A batch of frames
tail_length INT Number of frames to keep. Default: 17. Good values: 9, 17, 25.
Output Type Description
tail_frames IMAGE The extracted tail segment
actual_frame_count INT How many frames were actually extracted (may be less than requested)

Why 17? LTX works with 8-frame chunks + 1 overlap. A 17-frame tail gives 2 full chunks of motion context.

LTX Flow - First/Last Guide

Category: LTXFlow/LTX Guides

Injects first and last frame guidance into LTX conditioning using LTXVAddGuide internals.

Input Type Description
positive CONDITIONING Positive conditioning from the LTX text encoder
negative CONDITIONING Negative conditioning
vae VAE The LTX video VAE
latent LATENT Empty LTX latent
first_frame IMAGE Image to anchor at frame 0
last_frame IMAGE Image to anchor at the final frame
first_strength FLOAT Guidance strength for first frame (0.0–1.0)
last_strength FLOAT Guidance strength for last frame (0.0–1.0)
last_frame_index INT Frame position for last frame. -1 = final frame.
Output Type Description
positive CONDITIONING Updated positive conditioning
negative CONDITIONING Updated negative conditioning
latent LATENT Updated latent with noise mask

LTX Flow - Qwen Edit Bridge

Category: LTXFlow/Qwen

Prepares Qwen-edited images for the First/Last Guide by resizing to matching dimensions.

Input Type Description
source_image IMAGE Original source image
qwen_edited_image IMAGE Output from Qwen Image Edit
first_frame_source ENUM source_image or qwen_edited_image
resize_to ENUM Which image's size to use as the target
multiple_of INT Enforce dimension divisibility (default: 32)
Output Type Description
first_frame IMAGE Ready for First/Last Guide
last_frame IMAGE Ready for First/Last Guide
keyframe_batch IMAGE Both frames stacked as a 2-frame batch

LTX Flow - Tail Guide

Category: LTXFlow/LTX Guides

For Flow-style video extension. Takes the tail segment of a previous clip and uses it to condition the next clip's generation.

Input Type Description
tail_frames IMAGE Tail frames from Extract Tail
positive CONDITIONING LTX conditioning
negative CONDITIONING LTX conditioning
vae VAE LTX video VAE
latent LATENT Target latent for the new clip
tail_strength FLOAT How strongly the tail influences the next clip
Output Type Description
positive CONDITIONING Updated conditioning
negative CONDITIONING Updated conditioning
latent LATENT Updated latent with overlap
overlap_frames INT Number of frames that overlap with the previous clip

LTX Flow - Trim Frames

Category: LTXFlow/Frame Tools

Removes frames from the start or end of a clip. Needed after overlap-based extension to avoid duplicate frames.

LTX Flow - Scene Builder

Category: LTXFlow/Scene Tools

Concatenates up to 6 clips into a single frame batch for video export.

Input Type Description
clip_1 IMAGE First clip (required)
clip_2 through clip_6 IMAGE Additional clips (optional)
deduplicate_joins BOOLEAN Drop the first frame of each subsequent clip to avoid duplicates
fps INT Frames per second for duration calculation
Output Type Description
merged_frames IMAGE All clips concatenated
total_frames INT Total frame count
duration_seconds FLOAT Estimated video duration

Important: All clips must have the same height, width, and channel count. Resize them before merging if needed.

Building a Multi-Scene Video

The full pipeline for creating a multi-scene video:

Step 1: Generate Your First Clip

Use Workflow 05 to generate a Qwen-edited transition video from your source image.

Step 2: Extract the Tail

Feed the generated clip into Extract Tail with tail_length = 17.

Step 3: Generate the Next Clip

Use the Tail Guide node to condition the next LTX generation with the previous clip's tail. Write a new prompt for the continuation.

Step 4: Trim the Overlap

After generating the next clip, use Trim Frames to remove the overlapping frames at the start.

Step 5: Stitch Together

Feed all clips into Scene Builder:

clip_1 = first generated clip
clip_2 = second clip (trimmed)
clip_3 = third clip (trimmed)
deduplicate_joins = true
fps = 24

Step 6: Export

Send merged_frames to SaveVideo or VHS_VideoCombine to export your final video.

Recommended Settings

Qwen Image Edit

Parameter Recommended
Steps 40
CFG 4.0
Lightning LoRA steps 4

LTX Video 2.3

Parameter Recommended
Video length 5 seconds (start here)
First/Last strength 1.0 / 1.0
Upscale 2x on Stage 3 true
Tail length (for continuation) 17 frames

VRAM Management

VRAM Recommendation
24 GB Use fp8 models. Enable optimized decoding (tiled VAE). Keep video length ≤ 5 seconds.
48 GB Full models possible. Can extend to 8–10 second clips.
< 24 GB Not recommended. Consider cloud GPU (VastAI, RunPod).

Troubleshooting

"No outer link found for slot [0] image"

This means a node's image input isn't connected. Check that:

  • The LoadImage nodes have valid images selected
  • All wires between nodes are intact (click and drag to reconnect)
  • You're using the latest version of the workflow from this repo

"Installation Required" dialog on load

Install the missing custom node packs:

COMFYUI_DIR=/path/to/ComfyUI bash scripts/install_custom_nodes.sh

Or install through ComfyUI Manager.

Models not found in dropdowns

Make sure models are in the correct subdirectories:

ComfyUI/models/
├── checkpoints/
│   └── ltx-2.3-22b-dev-fp8.safetensors
├── diffusion_models/
│   └── qwen_image_edit_2511_bf16.safetensors
├── latent_upscale_models/
│   └── ltx-2.3-spatial-upscaler-x2-1.1.safetensors
├── loras/
│   ├── ltx-2.3-22b-distilled-lora-384.safetensors
│   └── Qwen-Image-Edit-2511-Lightning-4steps-V1.0-bf16.safetensors
├── text_encoders/
│   ├── gemma_3_12B_it_fp8_scaled.safetensors
│   └── qwen_2.5_vl_7b_fp8_scaled.safetensors
└── vae/
    └── qwen_image_vae.safetensors

Restart ComfyUI after placing models.

Video looks blurry or inconsistent

  • Enable Upscale 2x on Stage 3 in the sampler subgraph
  • Increase first_strength and last_strength to 1.0
  • Try a longer video (more frames = smoother interpolation)
  • Write a more detailed prompt

Out of VRAM

  • Enable Optimized Decoding (tiled VAE) in the decode subgraph
  • Reduce video length
  • Use the fp8 model variants (already included in the download script)
  • Close other GPU applications

Further Resources

Reference in New Issue View Git Blame Copy Permalink