chore(pyproject): added PublisherId to pyproject and docs for the two main nodes

This commit is contained in:
apenguin24
2025-10-24 19:43:24 +02:00
parent 2a46e757d8
commit 2a1cf4d0f5
3 changed files with 546 additions and 1 deletions
+1 -1
View File
@@ -28,7 +28,7 @@ Documentation = "https://github.com/apenguin24/ComfyUI-WanKeyframing/wiki"
"Bug Tracker" = "https://github.com/apenguin24/ComfyUI-WanKeyframing/issues"
[tool.comfy]
PublisherId = ""
PublisherId = "apenguin24"
DisplayName = "ComfyUI-WanKeyframing"
Icon = ""
includes = []
+344
View File
@@ -0,0 +1,344 @@
# LoadImagesFromDirectory
The LoadImagesFromDirectory node is a powerful batch image loader that efficiently processes multiple images from a directory with advanced resizing, sorting, and formatting options. This node is essential for keyframe-based workflows and batch processing tasks where consistent image dimensions and ordering are critical.
## Overview
This node loads all supported image files from a specified directory, applies consistent resizing, sorts them according to various criteria, and outputs them as a properly formatted batch tensor ready for use in ComfyUI workflows. It's particularly useful for creating keyframe sequences, batch processing workflows, and preparing image datasets.
## Key Features
- **Batch Loading**: Process entire directories of images in one operation
- **Smart Resizing**: Multiple resize modes to handle varying input dimensions
- **Flexible Sorting**: Sort images by name, date, size, or preserve original order
- **Format Support**: Wide range of image formats (PNG, JPG, JPEG, BMP, WEBP, TIF, TIFF)
- **Consistent Output**: All images normalized to identical dimensions and format
- **Reload Control**: Option to force refresh on each execution
- **Error Handling**: Robust error reporting for missing files or invalid formats
## Parameters
### Required Inputs
#### `directory` (STRING)
The directory path containing images to load, relative to ComfyUI's input directory.
**Path Resolution:**
- Path is relative to `ComfyUI/input/` directory
- Use forward slashes or backslashes as appropriate for your system
- Subdirectories are supported (e.g., `keyframes/sequence1`)
**Examples:**
```
"keyframes" → loads from ComfyUI/input/keyframes/
"project/frames" → loads from ComfyUI/input/project/frames/
"batch_001" → loads from ComfyUI/input/batch_001/
```
**Default:** `keyframes`
#### `reload_on_execute` (BOOLEAN)
Forces the node to reload images on every execution, bypassing ComfyUI's caching system.
**When to use:**
- **True**: When images in the directory change frequently during workflow development
- **False**: For stable directories where images don't change (better performance)
**Default:** `False`
#### `target_width` (INT)
Target width in pixels for all output images.
- **Range:** 64 - 8192 pixels
- **Step:** 8 pixels
- **Default:** 512
#### `target_height` (INT)
Target height in pixels for all output images.
- **Range:** 64 - 8192 pixels
- **Step:** 8 pixels
- **Default:** 512
#### `resize_mode` (ENUM)
Determines how images are resized to match target dimensions.
**Options:**
##### `stretch`
Directly stretches images to exact target dimensions.
- **Pros**: Exact dimensions guaranteed, simple processing
- **Cons**: May distort aspect ratio, can cause visual artifacts
- **Use case**: When exact dimensions are critical and aspect ratio changes are acceptable
##### `fit`
Maintains original aspect ratio, fits image inside target dimensions with padding.
- **Pros**: Preserves aspect ratio, no cropping
- **Cons**: May have black padding, doesn't fill entire frame
- **Use case**: When preserving entire image content is important
- **Padding**: Black padding added to center the image
##### `crop` (Default)
Maintains original aspect ratio, crops image to fill target dimensions.
- **Pros**: Fills entire frame, preserves aspect ratio
- **Cons**: May lose edge content through cropping
- **Use case**: When filling the frame completely is important
- **Cropping**: Intelligent center cropping to preserve main subject
**Default:** `crop`
#### `sort_mode` (ENUM)
Controls the order in which images are loaded and processed.
**Options:**
##### `name_asc` (Default)
Sort alphabetically by filename in ascending order (A-Z, 0-9).
- **Best for**: Numbered sequences (001.jpg, 002.jpg, etc.)
- **Example order**: frame_001.png, frame_002.png, frame_010.png
##### `name_desc`
Sort alphabetically by filename in descending order (Z-A, 9-0).
- **Best for**: Reverse sequences or reverse alphabetical order
##### `date_asc`
Sort by file modification date, oldest first.
- **Best for**: Processing images in chronological order of creation/modification
- **Useful for**: Photo sequences, time-lapse workflows
##### `date_desc`
Sort by file modification date, newest first.
- **Best for**: Processing most recent images first
##### `size_asc`
Sort by file size, smallest first.
- **Best for**: Processing lower resolution images before higher resolution
- **Useful for**: Progressive quality workflows
##### `size_desc`
Sort by file size, largest first.
- **Best for**: Processing highest quality images first
##### `none`
No sorting - uses system's default directory listing order.
- **Best for**: When order doesn't matter or preserving system order
- **Note**: Order may vary between operating systems
**Default:** `name_asc`
## Outputs
### `images` (IMAGE)
A batch tensor containing all loaded and processed images.
**Format:** `(batch_size, height, width, 3)`
- **batch_size**: Number of images loaded from directory
- **height**: Target height (as specified in parameters)
- **width**: Target width (as specified in parameters)
- **3**: RGB color channels
**Data Type:** `torch.FloatTensor`
**Value Range:** 0.0 - 1.0 (normalized from 0-255)
## Supported Image Formats
The node supports the following image formats:
- **PNG** (.png) - Lossless compression, transparency support
- **JPEG** (.jpg, .jpeg) - Lossy compression, smaller file sizes
- **BMP** (.bmp) - Uncompressed bitmap format
- **WebP** (.webp) - Modern web format with good compression
- **TIFF** (.tif, .tiff) - High quality, often used for professional photography
**Note:** All images are converted to RGB format regardless of input format, removing any alpha channels.
## Processing Pipeline
### 1. Directory Scanning
- Scans specified directory for supported image files
- Filters out hidden files (starting with '.')
- Validates file extensions against supported formats
### 2. File Sorting
- Applies selected sort mode to determine processing order
- Handles file system errors gracefully (places problematic files at end)
### 3. Image Loading & Processing
For each image:
- Opens image using PIL (Python Imaging Library)
- Converts to RGB format (removes alpha channels)
- Applies selected resize mode to reach target dimensions
- Converts to ComfyUI-compatible tensor format
- Normalizes values to 0.0-1.0 range
### 4. Batch Assembly
- Concatenates all processed images into a single batch tensor
- Ensures consistent dimensions across all images in batch
## Usage Examples
### Basic Keyframe Loading
```python
# Load numbered keyframes in order
directory = "my_keyframes"
sort_mode = "name_asc" # Ensures 001.jpg comes before 010.jpg
resize_mode = "crop" # Fill frame completely
target_width = 512
target_height = 512
```
### Time-lapse Processing
```python
# Process photos chronologically
directory = "timelapse_photos"
sort_mode = "date_asc" # Oldest photos first
resize_mode = "fit" # Preserve entire photo content
target_width = 1920
target_height = 1080
```
### Development Workflow
```python
# Frequently changing images during development
directory = "test_frames"
reload_on_execute = True # Always check for new/changed images
resize_mode = "stretch" # Quick processing for testing
target_width = 256 # Lower resolution for speed
target_height = 256
```
### High-Quality Processing
```python
# Process largest images first for quality workflows
directory = "high_res_sources"
sort_mode = "size_desc" # Largest files first
resize_mode = "crop" # Maintain quality, fill frame
target_width = 1024
target_height = 1024
```
## Resize Mode Comparison
| Mode | Aspect Ratio | Full Frame | Content Loss | Use Case |
|------|--------------|------------|--------------|----------|
| **stretch** | May change | ✅ Yes | Distortion | Exact dimensions needed |
| **fit** | ✅ Preserved | ❌ Padded | None | Preserve all content |
| **crop** | ✅ Preserved | ✅ Yes | Edge cropping | Fill frame, natural look |
## Performance Considerations
### Memory Usage
- Memory usage scales with: `batch_size × width × height × 3 × 4 bytes`
- For 100 images at 512×512: ~300MB RAM
- For 100 images at 1024×1024: ~1.2GB RAM
### Processing Speed
- **Fastest**: `stretch` mode (simple resize)
- **Medium**: `crop` mode (crop + resize)
- **Slowest**: `fit` mode (resize + composition)
### Optimization Tips
- Use `reload_on_execute = False` for stable directories
- Lower target dimensions for faster processing during development
- Consider batch size limits for available system memory
- Use appropriate resize mode for your specific needs
## Error Handling
### Common Errors and Solutions
#### `FileNotFoundError: Directory not found`
**Cause:** Specified directory doesn't exist relative to ComfyUI input folder
**Solution:**
- Verify directory path is correct
- Ensure directory exists in `ComfyUI/input/`
- Check for typos in directory name
#### `ValueError: No valid images found`
**Cause:** Directory exists but contains no supported image files
**Solution:**
- Verify images are in supported formats
- Check if images are hidden (filename starting with '.')
- Ensure files aren't corrupted
#### `RuntimeError: Failed to load image`
**Cause:** Specific image file is corrupted or in unsupported format
**Solution:**
- Check image file integrity
- Try opening image in standard image viewer
- Remove or replace corrupted files
- Verify file format is supported
#### Memory errors
**Cause:** Too many images or too high resolution for available RAM
**Solution:**
- Reduce target dimensions
- Process fewer images per batch
- Increase system RAM
- Use lower resolution images
## Integration Patterns
### Typical Workflow Connections
#### Keyframe Video Generation
```
LoadImagesFromDirectory → WanKeyframeToVideo
```
#### Batch Processing
```
LoadImagesFromDirectory → Image Processing Node → Save/Export
```
#### Style Transfer Pipeline
```
LoadImagesFromDirectory → Style Transfer → Batch Save
```
#### Dataset Preparation
```
LoadImagesFromDirectory → Preprocessing → Training Dataset
```
## Best Practices
### Directory Organization
- Use consistent naming conventions (001.jpg, 002.jpg, etc.)
- Keep related images in dedicated subdirectories
- Avoid spaces and special characters in filenames
- Use descriptive directory names
### Parameter Selection
- **For keyframes**: Use `name_asc` sorting with `crop` resize mode
- **For datasets**: Consider `date_asc` for chronological data
- **For development**: Enable `reload_on_execute` temporarily
- **For production**: Disable `reload_on_execute` for better performance
### File Management
- Regularly clean up temporary or test directories
- Keep original high-resolution sources separate from processed versions
- Consider using version control for important image sequences
- Backup critical image directories before batch processing
## Troubleshooting
### Images appear in wrong order
- Check `sort_mode` setting
- Verify filename numbering (use leading zeros: 001 not 1)
- Consider using `date_asc` for chronological ordering
### Images look stretched or distorted
- Switch from `stretch` to `crop` or `fit` mode
- Verify target dimensions match desired aspect ratio
- Check original image aspect ratios
### Memory issues with large batches
- Reduce `target_width` and `target_height`
- Process images in smaller batches
- Monitor system memory usage
- Consider using image compression
### Inconsistent results between runs
- Enable `reload_on_execute` if images change frequently
- Check for hidden files or temporary files in directory
- Verify all images are in supported formats
+201
View File
@@ -0,0 +1,201 @@
# WanKeyframeToVideo
The WanKeyframeToVideo node is an advanced keyframe-to-video encoder that creates smooth video sequences with intelligent conditioning and latent space interpolation. This node processes multiple keyframe pairs simultaneously to generate coherent video segments with segment-specific prompt conditioning.
## Overview
This node takes a sequence of keyframe images and generates video conditioning data that can be used with video generation models. It creates smooth transitions between keyframes while allowing different prompts to be applied to different segments of the video sequence.
## Key Features
- **Multi-Segment Processing**: Processes multiple keyframe pairs simultaneously for batch video generation
- **Indexed Prompt System**: Apply different prompts to different video segments using `[N]` indexing
- **CLIP Vision Integration**: Enhanced conditioning with optional vision embeddings
- **Flexible Video Control**: Precise control over FPS, duration, and dimensions
- **Smart Interpolation**: Intelligent latent space interpolation between keyframes
- **Batch Conditioning**: Efficiently stacks multiple segments into unified conditioning data
## Parameters
### Required Inputs
#### `clip` (CLIP)
The CLIP model used for text encoding. This generates text embeddings that condition the video generation process.
#### `positive_prompt` (STRING, multiline)
Multi-line text prompts with optional keyframe indexing. Use the format `[N] prompt text` to assign specific prompts to keyframe segments.
**Example:**
```
[0] A peaceful mountain meadow at sunrise, soft golden light
[1] The same meadow at midday, bright blue sky, vibrant colors
[2] Evening scene with purple sunset, dramatic lighting
```
**Default:** `[0] A beautiful meadow\n[1] A misty forest\n[2] A glowing futuristic city`
#### `negative_prompt` (STRING)
Negative conditioning text applied to all segments to specify what should be avoided in the generation.
**Default:** `low quality, blurry, bad lighting`
#### `vae` (VAE)
The Variational Autoencoder model used for encoding images to latent space and determining spatial compression ratios.
#### `width` (INT)
Target video width in pixels.
- **Range:** 16 - 8192 pixels
- **Step:** 16 pixels
- **Default:** 496
#### `height` (INT)
Target video height in pixels.
- **Range:** 16 - 8192 pixels
- **Step:** 16 pixels
- **Default:** 496
#### `fps` (INT)
Frames per second for the generated video.
- **Range:** 1 - 120 FPS
- **Default:** 16
#### `seconds` (INT)
Duration of each video segment in seconds.
- **Range:** 1 - 60 seconds
- **Default:** 1
### Optional Inputs
#### `keyframes` (IMAGE)
Input keyframe images that define the start and end points of video segments. The node requires at least 2 keyframes to create video segments. Each consecutive pair of keyframes creates one video segment.
**Requirements:**
- Minimum 2 keyframes required
- Images are automatically resized to match `width` and `height` parameters
- Supports batch input with multiple keyframes
#### `clip_vision_outputs` (CLIP_VISION_OUTPUT)
Optional CLIP vision embeddings for enhanced visual conditioning. When provided, these are merged with text conditioning to create more precise video generation guidance.
## Outputs
### `positive` (CONDITIONING)
Positive conditioning data containing:
- Stacked text embeddings from CLIP
- Pooled text outputs
- Concatenated latent images from keyframes
- Conditioning masks
- Optional CLIP vision embeddings
### `negative` (CONDITIONING)
Negative conditioning data with the same structure as positive conditioning, using the negative prompt for all segments.
### `latent` (LATENT)
Prepared latent tensor ready for video generation, with proper batch and temporal dimensions.
## How It Works
### 1. Prompt Processing
The node parses the positive prompt for indexed entries using the pattern `[N] prompt text`. Each index corresponds to a keyframe segment:
- `[0]` applies to the segment from keyframe 0 to keyframe 1
- `[1]` applies to the segment from keyframe 1 to keyframe 2
- And so on...
If no indexed prompts are found, the first line is used for all segments.
### 2. Keyframe Segmentation
With N keyframes, the node creates N-1 video segments. Each segment represents the transition from one keyframe to the next.
### 3. Frame Processing
For each segment:
- Start and end keyframes are extracted and resized to target dimensions
- A video tensor is created with the specified length (fps × seconds + 1)
- Start frame is placed at the beginning, end frame at the end
- Intermediate frames are filled with neutral values for interpolation
### 4. Conditioning Generation
Each segment gets:
- CLIP text encoding using the segment-specific prompt
- VAE latent encoding of the frame sequence
- Conditioning masks for proper temporal blending
- Optional CLIP vision processing if provided
### 5. Batch Stacking
All segment conditioning data is stacked into unified tensors that can be processed efficiently by video generation models.
## Usage Examples
### Basic Two-Keyframe Video
```python
# With 2 keyframes, creates 1 video segment
positive_prompt = "[0] A cat sitting in a garden, peaceful morning light"
# Segment 0: keyframe 0 → keyframe 1
```
### Multi-Segment Video
```python
# With 4 keyframes, creates 3 video segments
positive_prompt = """[0] A butterfly on a flower, macro photography
[1] The butterfly taking flight, motion blur, dynamic
[2] Wide shot of garden with butterfly in distance, aerial view"""
# Segment 0: keyframe 0 → keyframe 1 (macro butterfly)
# Segment 1: keyframe 1 → keyframe 2 (taking flight)
# Segment 2: keyframe 2 → keyframe 3 (wide garden view)
```
### Fallback Prompts
```python
# If keyframes don't have corresponding indexed prompts
positive_prompt = """[0] Default scene description
A forest path in different lighting conditions"""
# [0] used for segment 0, default text used for other segments
```
## Technical Notes
### Video Length Calculation
The actual video length is calculated as:
```python
length = (fps * seconds) + 1 # +1 for full second coverage
length = ((length - 1) // 4) * 4 + 1 # Round to multiple of 4 for compatibility
```
### Memory Considerations
- Video length directly impacts memory usage
- Higher resolutions require more VRAM
- More keyframes create more segments to process simultaneously
- Consider reducing parameters if encountering memory errors
### Compatibility
- Designed for video generation models that accept conditioning data
- Latent dimensions are automatically calculated based on VAE spatial compression
- Output tensors are formatted for standard ComfyUI video workflows
## Error Handling
### Common Issues
**"At least 2 keyframes are required"**
- Ensure your keyframes input contains at least 2 images
- Check that keyframes are properly connected in the workflow
**Memory errors**
- Reduce video resolution (`width`/`height`)
- Decrease `seconds` parameter
- Use fewer keyframes
- Lower `fps` setting
**Conditioning stacking warnings**
- These are usually non-critical and indicate fallback behavior
- The node will use the first available conditioning as backup
- Results should still be functional
## Integration
This node is typically used in video generation workflows:
1. **Load Images From Directory** → keyframes input
2. **CLIP Text Encoder** → clip input
3. **VAE Loader** → vae input
4. **WanKeyframeToVideo** → conditioning outputs
5. **Video Generation Model** (receives conditioning and latent outputs)