✨ (Dockerfile, docker-compose.yml): add httpx to requirements for enhanced HTTP requests support

✨ (Dockerfile, docker-compose.yml): update CMD to include --base-directory for better temp file management
📝 (TEMP_DIRECTORY_UTILS.md): add documentation for temporary directory utilities to explain usage and benefits
♻️ (temp_utils.py): create utility functions for ComfyUI-aware temporary file and directory management
♻️ (media_selection, media_describe, frame_extractor): refactor to use ComfyUI-aware temp utilities for better file handling
📝 (media-selection README, WORKFLOW_EXAMPLE.md, REDDIT_POST_MEDIA_SOURCE.md): update documentation to reflect changes in temp file management
💡 (media_selection): add comments to clarify the use of ComfyUI-aware temp directory functions
This commit is contained in:
Sam Kumar
2025-10-08 07:34:58 -05:00
parent 3052f3b97a
commit 570b178f85
10 changed files with 271 additions and 24 deletions
+3 -2
View File
@@ -35,7 +35,8 @@ RUN pip install --no-cache-dir -r requirements.txt
RUN pip install --no-cache-dir \
google-genai \
opencv-python \
requests
requests \
httpx
# Create directories for models and custom nodes if they don't exist
RUN mkdir -p models custom_nodes input output
@@ -51,4 +52,4 @@ RUN pip install --no-cache-dir -r custom_nodes/comfyui-manager/requirements.txt
EXPOSE 8188
# Default command (can be overridden in docker-compose.yml)
CMD ["python", "main.py", "--listen", "0.0.0.0", "--port", "8188", "--cpu"]
CMD ["python", "main.py", "--listen", "0.0.0.0", "--port", "8188", "--cpu", "--base-directory", "/workspace/ComfyUI"]
+2 -2
View File
@@ -4,7 +4,7 @@ services:
context: .
dockerfile: Dockerfile
ports:
- '8188:8188'
- "8188:8188"
volumes:
# Mount your custom node directory
- .:/workspace/ComfyUI/custom_nodes/ComfyUI-SwissArmyKnife
@@ -16,6 +16,6 @@ services:
- GEMINI_API_KEY=${GEMINI_API_KEY}
- CIVITAI_API_KEY=${CIVITAI_API_KEY}
working_dir: /workspace/ComfyUI
command: python main.py --listen 0.0.0.0 --port 8188 --cpu
command: python main.py --listen 0.0.0.0 --port 8188 --cpu --base-directory /workspace/ComfyUI
stdin_open: true
tty: true
+147
View File
@@ -0,0 +1,147 @@
# Temporary Directory Utilities
## Overview
ComfyUI-SwissArmyKnife uses a centralized temporary directory utility (`nodes/utils/temp_utils.py`) to ensure all temporary files and directories respect ComfyUI's configuration, especially in Docker environments with custom base directories.
## Why This Matters
### Problem with Standard Python tempfile
- `tempfile.gettempdir()` returns the system temp directory (e.g., `/tmp` on Linux)
- In Docker, this is the container's `/tmp`, not the ComfyUI workspace
- Files are isolated from the host and don't respect `--base-directory`
### Solution: ComfyUI-Aware Temp Directory
- Uses ComfyUI's `folder_paths.get_user_directory()` + `/temp`
- Respects `--base-directory` flag in Docker setups
- Falls back to system temp when ComfyUI not available
- Ensures consistent behavior across different deployment scenarios
## Usage
### Getting Base Temp Directory
```python
from ..utils.temp_utils import get_temp_directory
# Get ComfyUI's temp directory
temp_dir = get_temp_directory()
# Returns: /workspace/ComfyUI/user/temp (in Docker)
# Or: <user_dir>/temp (local install)
```
### Getting a Temp Subdirectory
```python
from ..utils.temp_utils import get_temp_subdirectory
# Create/get a subdirectory within temp
frames_dir = get_temp_subdirectory("comfyui_frames")
# Returns: /workspace/ComfyUI/user/temp/comfyui_frames
```
### Getting a Temp File Path
```python
from ..utils.temp_utils import get_temp_file_path
# Get a temp file path with specific extension
temp_video = get_temp_file_path(suffix='.mp4', prefix='trimmed', subdir='videos')
# Returns: /workspace/ComfyUI/user/temp/videos/trimmed_a1b2c3d4.mp4
# Use it like a regular file path
with open(temp_video, 'wb') as f:
f.write(video_data)
```
## Implementation Details
### Directory Structure
When using Docker with `--base-directory /workspace/ComfyUI`:
```
/workspace/ComfyUI/
├── user/
│ └── temp/ # Base temp directory
│ ├── comfyui_frames/ # Frame extraction subdirectory
│ │ └── frames_video_abc123/ # Per-video frame storage
│ ├── videos/ # Video processing subdirectory
│ │ └── trimmed_xyz789.mp4 # Temporary trimmed videos
│ └── downloads/ # Downloaded media subdirectory
│ └── reddit_media_def456.jpg
```
### Fallback Behavior
1. **ComfyUI Available**: Uses `folder_paths.get_user_directory() + '/temp'`
2. **ComfyUI Unavailable**: Falls back to `tempfile.gettempdir()`
This ensures compatibility with:
- Local ComfyUI installations
- Docker deployments with custom base directories
- Standalone testing/development environments
## Nodes Using Temp Utils
All nodes that create temporary files have been updated:
1. **Frame Extractor** (`nodes/media_selection/frame_extractor.py`)
- Uses `get_temp_subdirectory("comfyui_frames")` for frame extraction
2. **Media Selection** (`nodes/media_selection/media_selection.py`)
- Uses `get_temp_file_path(suffix='.mp4', subdir='videos')` for video trimming
3. **Media Describe** (`nodes/media_describe/mediia_describe.py`)
- Uses `get_temp_file_path(suffix=file_ext, subdir='downloads')` for Reddit downloads
- Uses `get_temp_file_path(suffix='.mp4', subdir='videos')` for video trimming
## Docker Configuration
In your `docker-compose.yml`:
```yaml
command: python main.py --listen 0.0.0.0 --port 8188 --cpu --base-directory /workspace/ComfyUI
```
With this setup:
- Temp files: `/workspace/ComfyUI/user/temp/...`
- Mounted to host: `./.comfyui/user/temp/...`
- Persistent across container restarts
## Benefits
✅ **Consistent behavior** across Docker and local installations
✅ **Respects ComfyUI configuration** (`--base-directory`, `--temp-directory`)
✅ **Better file organization** with subdirectories
✅ **Works in mounted volumes** (accessible from host)
✅ **Graceful fallback** when ComfyUI not available
## Migration from Python tempfile
### Before
```python
import tempfile
# Old way - uses system /tmp
base_temp_dir = tempfile.gettempdir()
# Or
with tempfile.NamedTemporaryFile(suffix='.mp4', delete=False) as temp_file:
temp_path = temp_file.name
```
### After
```python
from ..utils.temp_utils import get_temp_subdirectory, get_temp_file_path
# New way - uses ComfyUI temp directory
base_temp_dir = get_temp_subdirectory("my_subdir")
# Or
temp_path = get_temp_file_path(suffix='.mp4', prefix='my_prefix', subdir='my_subdir')
```
+1 -1
View File
@@ -262,7 +262,7 @@ Caption 3: She completes the turn with a graceful pose
### Frame Extraction
- Frames are saved to a temporary directory (`/tmp/frame_extract_*`)
- Frames are saved to a temporary directory within ComfyUI's temp directory (respects `--base-directory`)
- Frames are named with format: `frame_XXX_tY.YYs.png`
- PNG format preserves quality, JPG saves space
- Timestamps are calculated from frame number and FPS
@@ -42,11 +42,13 @@ The Frame Extractor outputs frame paths as a comma-separated string. You'll need
**Example output:**
```
/tmp/frame_extract_abc123/frame_000_t0.83s.png
/tmp/frame_extract_abc123/frame_001_t2.50s.png
/tmp/frame_extract_abc123/frame_002_t4.17s.png
<comfyui_temp>/comfyui_frames/frames_<video_filename>_<hash>/frame_000_t0.83s.png
<comfyui_temp>/comfyui_frames/frames_<video_filename>_<hash>/frame_001_t2.50s.png
<comfyui_temp>/comfyui_frames/frames_<video_filename>_<hash>/frame_002_t4.17s.png
```
**Note**: The temp directory respects ComfyUI's `--base-directory` and `--temp-directory` flags.
### Step 4: Process Frames with JoyCaption
For each frame:
@@ -136,7 +136,7 @@ New imports added to `nodes/nodes.py`:
### Temporary File Management
- Media files are downloaded to temporary files using `tempfile.NamedTemporaryFile`
- Media files are downloaded to ComfyUI's temp directory (respects `--base-directory` and `--temp-directory` flags)
- Files are not automatically deleted (delete=False) to allow processing
- Temporary files should be cleaned up by the system or calling code
+6 -7
View File
@@ -1,5 +1,4 @@
import cv2
import tempfile
import numpy as np
from PIL import Image
import mimetypes
@@ -18,6 +17,7 @@ from google import genai
from google.genai import types
from ..cache import get_cache, get_file_media_identifier, get_tensor_media_identifier
from ..utils.temp_utils import get_temp_file_path
class MediaDescribe:
"""
@@ -609,10 +609,10 @@ class MediaDescribe:
if not file_ext:
file_ext = '.mp4' if media_type == 'video' else '.jpg'
# Create temporary file
with tempfile.NamedTemporaryFile(suffix=file_ext, delete=False) as temp_file:
# Create temporary file using ComfyUI-aware temp directory
temp_path = get_temp_file_path(suffix=file_ext, prefix='reddit_media', subdir='downloads')
with open(temp_path, 'wb') as temp_file:
temp_file.write(media_response.content)
temp_path = temp_file.name
# Create media info
file_size = len(media_response.content)
@@ -1388,9 +1388,8 @@ Example (structure only):
# Check if we need to trim the video (only duration limit)
if max_duration > 0 and actual_duration < original_duration:
# Create a temporary trimmed video file
with tempfile.NamedTemporaryFile(suffix='.mp4', delete=False) as temp_file:
trimmed_video_path = temp_file.name
# Create a temporary trimmed video file using ComfyUI-aware temp directory
trimmed_video_path = get_temp_file_path(suffix='.mp4', prefix='trimmed', subdir='videos')
# Attempt to trim the video
print(f"Attempting to trim video from {original_duration:.2f}s to {actual_duration:.2f}s")
+6 -5
View File
@@ -1,10 +1,10 @@
import cv2
import os
import random
import tempfile
import numpy as np
from PIL import Image
import hashlib
from ..utils.temp_utils import get_temp_subdirectory
class FrameExtractor:
@@ -186,10 +186,8 @@ class FrameExtractor:
"""
Create a uniquely named directory for frames based on the video file.
Uses video filename and a hash of the full path to ensure uniqueness.
Uses ComfyUI's temp directory when available, respecting --base-directory.
"""
# Get base temp directory
base_temp_dir = tempfile.gettempdir()
# Extract video filename without extension
video_filename = os.path.splitext(os.path.basename(video_path))[0]
@@ -198,7 +196,10 @@ class FrameExtractor:
# Create directory name: video_filename_hash
dir_name = f"frames_{video_filename}_{path_hash}"
frames_dir = os.path.join(base_temp_dir, "comfyui_frames", dir_name)
# Use ComfyUI-aware temp directory
base_temp_dir = get_temp_subdirectory("comfyui_frames")
frames_dir = os.path.join(base_temp_dir, dir_name)
# Create directory if it doesn't exist
os.makedirs(frames_dir, exist_ok=True)
+3 -3
View File
@@ -2,12 +2,12 @@ import cv2
import os
import random
import glob
import tempfile
import requests
import mimetypes
from urllib.parse import urlparse
from html import unescape
import subprocess
from ..utils.temp_utils import get_temp_file_path
class MediaSelection:
@@ -277,8 +277,8 @@ class MediaSelection:
min_duration = min(1.0, original_duration)
actual_duration = max(min_duration, min(max_duration, original_duration))
with tempfile.NamedTemporaryFile(suffix='.mp4', delete=False) as temp_file:
trimmed_video_path = temp_file.name
# Use ComfyUI-aware temp directory
trimmed_video_path = get_temp_file_path(suffix='.mp4', prefix='trimmed', subdir='videos')
print(f"Attempting to trim video from {original_duration:.2f}s to {actual_duration:.2f}s")
if self._trim_video(video_path, trimmed_video_path, actual_duration):
+97
View File
@@ -0,0 +1,97 @@
"""
Temporary directory utilities for ComfyUI-SwissArmyKnife.
Provides ComfyUI-aware temporary file and directory management that respects
the --base-directory and --temp-directory flags.
"""
import os
import tempfile
from typing import Optional
try:
import folder_paths
COMFYUI_AVAILABLE = True
except ImportError:
folder_paths = None
COMFYUI_AVAILABLE = False
def get_temp_directory() -> str:
"""
Get the base temporary directory for ComfyUI operations.
This function respects ComfyUI's configuration:
- Uses ComfyUI's user directory + '/temp' if available
- Falls back to system temp directory if ComfyUI not available
When using Docker with --base-directory, this ensures temp files
are stored within the ComfyUI workspace rather than the container's /tmp.
Returns:
str: Path to the base temporary directory
"""
if COMFYUI_AVAILABLE and folder_paths is not None:
try:
# Use ComfyUI's user directory for temp files
user_dir = folder_paths.get_user_directory()
temp_dir = os.path.join(user_dir, "temp")
os.makedirs(temp_dir, exist_ok=True)
return temp_dir
except Exception as e:
print(f"Warning: Could not get ComfyUI temp directory: {e}")
# Fallback to system temp directory
return tempfile.gettempdir()
def get_temp_subdirectory(subdir_name: str) -> str:
"""
Get or create a subdirectory within ComfyUI's temp directory.
Args:
subdir_name: Name of the subdirectory to create
Returns:
str: Path to the subdirectory
Example:
frames_dir = get_temp_subdirectory("comfyui_frames")
"""
base_temp = get_temp_directory()
subdir = os.path.join(base_temp, subdir_name)
os.makedirs(subdir, exist_ok=True)
return subdir
def get_temp_file_path(suffix: str = "", prefix: str = "tmp", subdir: Optional[str] = None) -> str:
"""
Generate a temporary file path within ComfyUI's temp directory.
This is a replacement for tempfile.NamedTemporaryFile that respects
ComfyUI's base directory configuration.
Args:
suffix: File extension (e.g., '.mp4', '.jpg')
prefix: Filename prefix (default: 'tmp')
subdir: Optional subdirectory within temp (e.g., 'video_processing')
Returns:
str: Full path to the temporary file (file is NOT created)
Example:
temp_video = get_temp_file_path(suffix='.mp4', subdir='videos')
# Use the path to write your file
with open(temp_video, 'wb') as f:
f.write(video_data)
"""
if subdir:
base_temp = get_temp_subdirectory(subdir)
else:
base_temp = get_temp_directory()
# Generate a unique filename
import uuid
unique_id = uuid.uuid4().hex[:8]
filename = f"{prefix}_{unique_id}{suffix}"
return os.path.join(base_temp, filename)