chore(pyproject): added PublisherId to pyproject and docs for the two main nodes
This commit is contained in:
+1
-1
@@ -28,7 +28,7 @@ Documentation = "https://github.com/apenguin24/ComfyUI-WanKeyframing/wiki"
|
||||
"Bug Tracker" = "https://github.com/apenguin24/ComfyUI-WanKeyframing/issues"
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = ""
|
||||
PublisherId = "apenguin24"
|
||||
DisplayName = "ComfyUI-WanKeyframing"
|
||||
Icon = ""
|
||||
includes = []
|
||||
|
||||
@@ -0,0 +1,344 @@
|
||||
# LoadImagesFromDirectory
|
||||
|
||||
The LoadImagesFromDirectory node is a powerful batch image loader that efficiently processes multiple images from a directory with advanced resizing, sorting, and formatting options. This node is essential for keyframe-based workflows and batch processing tasks where consistent image dimensions and ordering are critical.
|
||||
|
||||
## Overview
|
||||
|
||||
This node loads all supported image files from a specified directory, applies consistent resizing, sorts them according to various criteria, and outputs them as a properly formatted batch tensor ready for use in ComfyUI workflows. It's particularly useful for creating keyframe sequences, batch processing workflows, and preparing image datasets.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Batch Loading**: Process entire directories of images in one operation
|
||||
- **Smart Resizing**: Multiple resize modes to handle varying input dimensions
|
||||
- **Flexible Sorting**: Sort images by name, date, size, or preserve original order
|
||||
- **Format Support**: Wide range of image formats (PNG, JPG, JPEG, BMP, WEBP, TIF, TIFF)
|
||||
- **Consistent Output**: All images normalized to identical dimensions and format
|
||||
- **Reload Control**: Option to force refresh on each execution
|
||||
- **Error Handling**: Robust error reporting for missing files or invalid formats
|
||||
|
||||
## Parameters
|
||||
|
||||
### Required Inputs
|
||||
|
||||
#### `directory` (STRING)
|
||||
The directory path containing images to load, relative to ComfyUI's input directory.
|
||||
|
||||
**Path Resolution:**
|
||||
- Path is relative to `ComfyUI/input/` directory
|
||||
- Use forward slashes or backslashes as appropriate for your system
|
||||
- Subdirectories are supported (e.g., `keyframes/sequence1`)
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
"keyframes" → loads from ComfyUI/input/keyframes/
|
||||
"project/frames" → loads from ComfyUI/input/project/frames/
|
||||
"batch_001" → loads from ComfyUI/input/batch_001/
|
||||
```
|
||||
|
||||
**Default:** `keyframes`
|
||||
|
||||
#### `reload_on_execute` (BOOLEAN)
|
||||
Forces the node to reload images on every execution, bypassing ComfyUI's caching system.
|
||||
|
||||
**When to use:**
|
||||
- **True**: When images in the directory change frequently during workflow development
|
||||
- **False**: For stable directories where images don't change (better performance)
|
||||
|
||||
**Default:** `False`
|
||||
|
||||
#### `target_width` (INT)
|
||||
Target width in pixels for all output images.
|
||||
|
||||
- **Range:** 64 - 8192 pixels
|
||||
- **Step:** 8 pixels
|
||||
- **Default:** 512
|
||||
|
||||
#### `target_height` (INT)
|
||||
Target height in pixels for all output images.
|
||||
|
||||
- **Range:** 64 - 8192 pixels
|
||||
- **Step:** 8 pixels
|
||||
- **Default:** 512
|
||||
|
||||
#### `resize_mode` (ENUM)
|
||||
Determines how images are resized to match target dimensions.
|
||||
|
||||
**Options:**
|
||||
|
||||
##### `stretch`
|
||||
Directly stretches images to exact target dimensions.
|
||||
- **Pros**: Exact dimensions guaranteed, simple processing
|
||||
- **Cons**: May distort aspect ratio, can cause visual artifacts
|
||||
- **Use case**: When exact dimensions are critical and aspect ratio changes are acceptable
|
||||
|
||||
##### `fit`
|
||||
Maintains original aspect ratio, fits image inside target dimensions with padding.
|
||||
- **Pros**: Preserves aspect ratio, no cropping
|
||||
- **Cons**: May have black padding, doesn't fill entire frame
|
||||
- **Use case**: When preserving entire image content is important
|
||||
- **Padding**: Black padding added to center the image
|
||||
|
||||
##### `crop` (Default)
|
||||
Maintains original aspect ratio, crops image to fill target dimensions.
|
||||
- **Pros**: Fills entire frame, preserves aspect ratio
|
||||
- **Cons**: May lose edge content through cropping
|
||||
- **Use case**: When filling the frame completely is important
|
||||
- **Cropping**: Intelligent center cropping to preserve main subject
|
||||
|
||||
**Default:** `crop`
|
||||
|
||||
#### `sort_mode` (ENUM)
|
||||
Controls the order in which images are loaded and processed.
|
||||
|
||||
**Options:**
|
||||
|
||||
##### `name_asc` (Default)
|
||||
Sort alphabetically by filename in ascending order (A-Z, 0-9).
|
||||
- **Best for**: Numbered sequences (001.jpg, 002.jpg, etc.)
|
||||
- **Example order**: frame_001.png, frame_002.png, frame_010.png
|
||||
|
||||
##### `name_desc`
|
||||
Sort alphabetically by filename in descending order (Z-A, 9-0).
|
||||
- **Best for**: Reverse sequences or reverse alphabetical order
|
||||
|
||||
##### `date_asc`
|
||||
Sort by file modification date, oldest first.
|
||||
- **Best for**: Processing images in chronological order of creation/modification
|
||||
- **Useful for**: Photo sequences, time-lapse workflows
|
||||
|
||||
##### `date_desc`
|
||||
Sort by file modification date, newest first.
|
||||
- **Best for**: Processing most recent images first
|
||||
|
||||
##### `size_asc`
|
||||
Sort by file size, smallest first.
|
||||
- **Best for**: Processing lower resolution images before higher resolution
|
||||
- **Useful for**: Progressive quality workflows
|
||||
|
||||
##### `size_desc`
|
||||
Sort by file size, largest first.
|
||||
- **Best for**: Processing highest quality images first
|
||||
|
||||
##### `none`
|
||||
No sorting - uses system's default directory listing order.
|
||||
- **Best for**: When order doesn't matter or preserving system order
|
||||
- **Note**: Order may vary between operating systems
|
||||
|
||||
**Default:** `name_asc`
|
||||
|
||||
## Outputs
|
||||
|
||||
### `images` (IMAGE)
|
||||
A batch tensor containing all loaded and processed images.
|
||||
|
||||
**Format:** `(batch_size, height, width, 3)`
|
||||
- **batch_size**: Number of images loaded from directory
|
||||
- **height**: Target height (as specified in parameters)
|
||||
- **width**: Target width (as specified in parameters)
|
||||
- **3**: RGB color channels
|
||||
|
||||
**Data Type:** `torch.FloatTensor`
|
||||
**Value Range:** 0.0 - 1.0 (normalized from 0-255)
|
||||
|
||||
## Supported Image Formats
|
||||
|
||||
The node supports the following image formats:
|
||||
- **PNG** (.png) - Lossless compression, transparency support
|
||||
- **JPEG** (.jpg, .jpeg) - Lossy compression, smaller file sizes
|
||||
- **BMP** (.bmp) - Uncompressed bitmap format
|
||||
- **WebP** (.webp) - Modern web format with good compression
|
||||
- **TIFF** (.tif, .tiff) - High quality, often used for professional photography
|
||||
|
||||
**Note:** All images are converted to RGB format regardless of input format, removing any alpha channels.
|
||||
|
||||
## Processing Pipeline
|
||||
|
||||
### 1. Directory Scanning
|
||||
- Scans specified directory for supported image files
|
||||
- Filters out hidden files (starting with '.')
|
||||
- Validates file extensions against supported formats
|
||||
|
||||
### 2. File Sorting
|
||||
- Applies selected sort mode to determine processing order
|
||||
- Handles file system errors gracefully (places problematic files at end)
|
||||
|
||||
### 3. Image Loading & Processing
|
||||
For each image:
|
||||
- Opens image using PIL (Python Imaging Library)
|
||||
- Converts to RGB format (removes alpha channels)
|
||||
- Applies selected resize mode to reach target dimensions
|
||||
- Converts to ComfyUI-compatible tensor format
|
||||
- Normalizes values to 0.0-1.0 range
|
||||
|
||||
### 4. Batch Assembly
|
||||
- Concatenates all processed images into a single batch tensor
|
||||
- Ensures consistent dimensions across all images in batch
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Keyframe Loading
|
||||
```python
|
||||
# Load numbered keyframes in order
|
||||
directory = "my_keyframes"
|
||||
sort_mode = "name_asc" # Ensures 001.jpg comes before 010.jpg
|
||||
resize_mode = "crop" # Fill frame completely
|
||||
target_width = 512
|
||||
target_height = 512
|
||||
```
|
||||
|
||||
### Time-lapse Processing
|
||||
```python
|
||||
# Process photos chronologically
|
||||
directory = "timelapse_photos"
|
||||
sort_mode = "date_asc" # Oldest photos first
|
||||
resize_mode = "fit" # Preserve entire photo content
|
||||
target_width = 1920
|
||||
target_height = 1080
|
||||
```
|
||||
|
||||
### Development Workflow
|
||||
```python
|
||||
# Frequently changing images during development
|
||||
directory = "test_frames"
|
||||
reload_on_execute = True # Always check for new/changed images
|
||||
resize_mode = "stretch" # Quick processing for testing
|
||||
target_width = 256 # Lower resolution for speed
|
||||
target_height = 256
|
||||
```
|
||||
|
||||
### High-Quality Processing
|
||||
```python
|
||||
# Process largest images first for quality workflows
|
||||
directory = "high_res_sources"
|
||||
sort_mode = "size_desc" # Largest files first
|
||||
resize_mode = "crop" # Maintain quality, fill frame
|
||||
target_width = 1024
|
||||
target_height = 1024
|
||||
```
|
||||
|
||||
## Resize Mode Comparison
|
||||
|
||||
| Mode | Aspect Ratio | Full Frame | Content Loss | Use Case |
|
||||
|------|--------------|------------|--------------|----------|
|
||||
| **stretch** | May change | ✅ Yes | Distortion | Exact dimensions needed |
|
||||
| **fit** | ✅ Preserved | ❌ Padded | None | Preserve all content |
|
||||
| **crop** | ✅ Preserved | ✅ Yes | Edge cropping | Fill frame, natural look |
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Memory Usage
|
||||
- Memory usage scales with: `batch_size × width × height × 3 × 4 bytes`
|
||||
- For 100 images at 512×512: ~300MB RAM
|
||||
- For 100 images at 1024×1024: ~1.2GB RAM
|
||||
|
||||
### Processing Speed
|
||||
- **Fastest**: `stretch` mode (simple resize)
|
||||
- **Medium**: `crop` mode (crop + resize)
|
||||
- **Slowest**: `fit` mode (resize + composition)
|
||||
|
||||
### Optimization Tips
|
||||
- Use `reload_on_execute = False` for stable directories
|
||||
- Lower target dimensions for faster processing during development
|
||||
- Consider batch size limits for available system memory
|
||||
- Use appropriate resize mode for your specific needs
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Errors and Solutions
|
||||
|
||||
#### `FileNotFoundError: Directory not found`
|
||||
**Cause:** Specified directory doesn't exist relative to ComfyUI input folder
|
||||
**Solution:**
|
||||
- Verify directory path is correct
|
||||
- Ensure directory exists in `ComfyUI/input/`
|
||||
- Check for typos in directory name
|
||||
|
||||
#### `ValueError: No valid images found`
|
||||
**Cause:** Directory exists but contains no supported image files
|
||||
**Solution:**
|
||||
- Verify images are in supported formats
|
||||
- Check if images are hidden (filename starting with '.')
|
||||
- Ensure files aren't corrupted
|
||||
|
||||
#### `RuntimeError: Failed to load image`
|
||||
**Cause:** Specific image file is corrupted or in unsupported format
|
||||
**Solution:**
|
||||
- Check image file integrity
|
||||
- Try opening image in standard image viewer
|
||||
- Remove or replace corrupted files
|
||||
- Verify file format is supported
|
||||
|
||||
#### Memory errors
|
||||
**Cause:** Too many images or too high resolution for available RAM
|
||||
**Solution:**
|
||||
- Reduce target dimensions
|
||||
- Process fewer images per batch
|
||||
- Increase system RAM
|
||||
- Use lower resolution images
|
||||
|
||||
## Integration Patterns
|
||||
|
||||
### Typical Workflow Connections
|
||||
|
||||
#### Keyframe Video Generation
|
||||
```
|
||||
LoadImagesFromDirectory → WanKeyframeToVideo
|
||||
```
|
||||
|
||||
#### Batch Processing
|
||||
```
|
||||
LoadImagesFromDirectory → Image Processing Node → Save/Export
|
||||
```
|
||||
|
||||
#### Style Transfer Pipeline
|
||||
```
|
||||
LoadImagesFromDirectory → Style Transfer → Batch Save
|
||||
```
|
||||
|
||||
#### Dataset Preparation
|
||||
```
|
||||
LoadImagesFromDirectory → Preprocessing → Training Dataset
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Directory Organization
|
||||
- Use consistent naming conventions (001.jpg, 002.jpg, etc.)
|
||||
- Keep related images in dedicated subdirectories
|
||||
- Avoid spaces and special characters in filenames
|
||||
- Use descriptive directory names
|
||||
|
||||
### Parameter Selection
|
||||
- **For keyframes**: Use `name_asc` sorting with `crop` resize mode
|
||||
- **For datasets**: Consider `date_asc` for chronological data
|
||||
- **For development**: Enable `reload_on_execute` temporarily
|
||||
- **For production**: Disable `reload_on_execute` for better performance
|
||||
|
||||
### File Management
|
||||
- Regularly clean up temporary or test directories
|
||||
- Keep original high-resolution sources separate from processed versions
|
||||
- Consider using version control for important image sequences
|
||||
- Backup critical image directories before batch processing
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Images appear in wrong order
|
||||
- Check `sort_mode` setting
|
||||
- Verify filename numbering (use leading zeros: 001 not 1)
|
||||
- Consider using `date_asc` for chronological ordering
|
||||
|
||||
### Images look stretched or distorted
|
||||
- Switch from `stretch` to `crop` or `fit` mode
|
||||
- Verify target dimensions match desired aspect ratio
|
||||
- Check original image aspect ratios
|
||||
|
||||
### Memory issues with large batches
|
||||
- Reduce `target_width` and `target_height`
|
||||
- Process images in smaller batches
|
||||
- Monitor system memory usage
|
||||
- Consider using image compression
|
||||
|
||||
### Inconsistent results between runs
|
||||
- Enable `reload_on_execute` if images change frequently
|
||||
- Check for hidden files or temporary files in directory
|
||||
- Verify all images are in supported formats
|
||||
@@ -0,0 +1,201 @@
|
||||
# WanKeyframeToVideo
|
||||
|
||||
The WanKeyframeToVideo node is an advanced keyframe-to-video encoder that creates smooth video sequences with intelligent conditioning and latent space interpolation. This node processes multiple keyframe pairs simultaneously to generate coherent video segments with segment-specific prompt conditioning.
|
||||
|
||||
## Overview
|
||||
|
||||
This node takes a sequence of keyframe images and generates video conditioning data that can be used with video generation models. It creates smooth transitions between keyframes while allowing different prompts to be applied to different segments of the video sequence.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Multi-Segment Processing**: Processes multiple keyframe pairs simultaneously for batch video generation
|
||||
- **Indexed Prompt System**: Apply different prompts to different video segments using `[N]` indexing
|
||||
- **CLIP Vision Integration**: Enhanced conditioning with optional vision embeddings
|
||||
- **Flexible Video Control**: Precise control over FPS, duration, and dimensions
|
||||
- **Smart Interpolation**: Intelligent latent space interpolation between keyframes
|
||||
- **Batch Conditioning**: Efficiently stacks multiple segments into unified conditioning data
|
||||
|
||||
## Parameters
|
||||
|
||||
### Required Inputs
|
||||
|
||||
#### `clip` (CLIP)
|
||||
The CLIP model used for text encoding. This generates text embeddings that condition the video generation process.
|
||||
|
||||
#### `positive_prompt` (STRING, multiline)
|
||||
Multi-line text prompts with optional keyframe indexing. Use the format `[N] prompt text` to assign specific prompts to keyframe segments.
|
||||
|
||||
**Example:**
|
||||
```
|
||||
[0] A peaceful mountain meadow at sunrise, soft golden light
|
||||
[1] The same meadow at midday, bright blue sky, vibrant colors
|
||||
[2] Evening scene with purple sunset, dramatic lighting
|
||||
```
|
||||
|
||||
**Default:** `[0] A beautiful meadow\n[1] A misty forest\n[2] A glowing futuristic city`
|
||||
|
||||
#### `negative_prompt` (STRING)
|
||||
Negative conditioning text applied to all segments to specify what should be avoided in the generation.
|
||||
|
||||
**Default:** `low quality, blurry, bad lighting`
|
||||
|
||||
#### `vae` (VAE)
|
||||
The Variational Autoencoder model used for encoding images to latent space and determining spatial compression ratios.
|
||||
|
||||
#### `width` (INT)
|
||||
Target video width in pixels.
|
||||
- **Range:** 16 - 8192 pixels
|
||||
- **Step:** 16 pixels
|
||||
- **Default:** 496
|
||||
|
||||
#### `height` (INT)
|
||||
Target video height in pixels.
|
||||
- **Range:** 16 - 8192 pixels
|
||||
- **Step:** 16 pixels
|
||||
- **Default:** 496
|
||||
|
||||
#### `fps` (INT)
|
||||
Frames per second for the generated video.
|
||||
- **Range:** 1 - 120 FPS
|
||||
- **Default:** 16
|
||||
|
||||
#### `seconds` (INT)
|
||||
Duration of each video segment in seconds.
|
||||
- **Range:** 1 - 60 seconds
|
||||
- **Default:** 1
|
||||
|
||||
### Optional Inputs
|
||||
|
||||
#### `keyframes` (IMAGE)
|
||||
Input keyframe images that define the start and end points of video segments. The node requires at least 2 keyframes to create video segments. Each consecutive pair of keyframes creates one video segment.
|
||||
|
||||
**Requirements:**
|
||||
- Minimum 2 keyframes required
|
||||
- Images are automatically resized to match `width` and `height` parameters
|
||||
- Supports batch input with multiple keyframes
|
||||
|
||||
#### `clip_vision_outputs` (CLIP_VISION_OUTPUT)
|
||||
Optional CLIP vision embeddings for enhanced visual conditioning. When provided, these are merged with text conditioning to create more precise video generation guidance.
|
||||
|
||||
## Outputs
|
||||
|
||||
### `positive` (CONDITIONING)
|
||||
Positive conditioning data containing:
|
||||
- Stacked text embeddings from CLIP
|
||||
- Pooled text outputs
|
||||
- Concatenated latent images from keyframes
|
||||
- Conditioning masks
|
||||
- Optional CLIP vision embeddings
|
||||
|
||||
### `negative` (CONDITIONING)
|
||||
Negative conditioning data with the same structure as positive conditioning, using the negative prompt for all segments.
|
||||
|
||||
### `latent` (LATENT)
|
||||
Prepared latent tensor ready for video generation, with proper batch and temporal dimensions.
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Prompt Processing
|
||||
The node parses the positive prompt for indexed entries using the pattern `[N] prompt text`. Each index corresponds to a keyframe segment:
|
||||
- `[0]` applies to the segment from keyframe 0 to keyframe 1
|
||||
- `[1]` applies to the segment from keyframe 1 to keyframe 2
|
||||
- And so on...
|
||||
|
||||
If no indexed prompts are found, the first line is used for all segments.
|
||||
|
||||
### 2. Keyframe Segmentation
|
||||
With N keyframes, the node creates N-1 video segments. Each segment represents the transition from one keyframe to the next.
|
||||
|
||||
### 3. Frame Processing
|
||||
For each segment:
|
||||
- Start and end keyframes are extracted and resized to target dimensions
|
||||
- A video tensor is created with the specified length (fps × seconds + 1)
|
||||
- Start frame is placed at the beginning, end frame at the end
|
||||
- Intermediate frames are filled with neutral values for interpolation
|
||||
|
||||
### 4. Conditioning Generation
|
||||
Each segment gets:
|
||||
- CLIP text encoding using the segment-specific prompt
|
||||
- VAE latent encoding of the frame sequence
|
||||
- Conditioning masks for proper temporal blending
|
||||
- Optional CLIP vision processing if provided
|
||||
|
||||
### 5. Batch Stacking
|
||||
All segment conditioning data is stacked into unified tensors that can be processed efficiently by video generation models.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Two-Keyframe Video
|
||||
```python
|
||||
# With 2 keyframes, creates 1 video segment
|
||||
positive_prompt = "[0] A cat sitting in a garden, peaceful morning light"
|
||||
# Segment 0: keyframe 0 → keyframe 1
|
||||
```
|
||||
|
||||
### Multi-Segment Video
|
||||
```python
|
||||
# With 4 keyframes, creates 3 video segments
|
||||
positive_prompt = """[0] A butterfly on a flower, macro photography
|
||||
[1] The butterfly taking flight, motion blur, dynamic
|
||||
[2] Wide shot of garden with butterfly in distance, aerial view"""
|
||||
# Segment 0: keyframe 0 → keyframe 1 (macro butterfly)
|
||||
# Segment 1: keyframe 1 → keyframe 2 (taking flight)
|
||||
# Segment 2: keyframe 2 → keyframe 3 (wide garden view)
|
||||
```
|
||||
|
||||
### Fallback Prompts
|
||||
```python
|
||||
# If keyframes don't have corresponding indexed prompts
|
||||
positive_prompt = """[0] Default scene description
|
||||
A forest path in different lighting conditions"""
|
||||
# [0] used for segment 0, default text used for other segments
|
||||
```
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### Video Length Calculation
|
||||
The actual video length is calculated as:
|
||||
```python
|
||||
length = (fps * seconds) + 1 # +1 for full second coverage
|
||||
length = ((length - 1) // 4) * 4 + 1 # Round to multiple of 4 for compatibility
|
||||
```
|
||||
|
||||
### Memory Considerations
|
||||
- Video length directly impacts memory usage
|
||||
- Higher resolutions require more VRAM
|
||||
- More keyframes create more segments to process simultaneously
|
||||
- Consider reducing parameters if encountering memory errors
|
||||
|
||||
### Compatibility
|
||||
- Designed for video generation models that accept conditioning data
|
||||
- Latent dimensions are automatically calculated based on VAE spatial compression
|
||||
- Output tensors are formatted for standard ComfyUI video workflows
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Issues
|
||||
|
||||
**"At least 2 keyframes are required"**
|
||||
- Ensure your keyframes input contains at least 2 images
|
||||
- Check that keyframes are properly connected in the workflow
|
||||
|
||||
**Memory errors**
|
||||
- Reduce video resolution (`width`/`height`)
|
||||
- Decrease `seconds` parameter
|
||||
- Use fewer keyframes
|
||||
- Lower `fps` setting
|
||||
|
||||
**Conditioning stacking warnings**
|
||||
- These are usually non-critical and indicate fallback behavior
|
||||
- The node will use the first available conditioning as backup
|
||||
- Results should still be functional
|
||||
|
||||
## Integration
|
||||
|
||||
This node is typically used in video generation workflows:
|
||||
|
||||
1. **Load Images From Directory** → keyframes input
|
||||
2. **CLIP Text Encoder** → clip input
|
||||
3. **VAE Loader** → vae input
|
||||
4. **WanKeyframeToVideo** → conditioning outputs
|
||||
5. **Video Generation Model** (receives conditioning and latent outputs)
|
||||
Reference in New Issue
Block a user