Key updates:
Added AddFramesToVideoFolder node to add frames to a video folder
Introduced VideoFolderProvider and VideoDefinitionProvider nodes to manage video folders and video definitions
Added FadeToInVideoFolder node for fading between frames in a video folder
Implemented ListedSVDConditioning node for applying SVD conditioning to a list of parameters
Added VideoFolderFFMPEGVideoInterpolate node to interpolate frames in a video folder using FFmpeg
Introduced AudioFolderProvider and AudioDefinitionProvider nodes for managing audio folders and definitions
Added AddAudioToAudioFolder and AddSilenceToAudioFolder nodes for adding audio files and silence to audio folders
Implemented GetAudioFilesLengthsFromAudioFolder node to retrieve lengths of audio files in a folder
Added AudioFolderRenderAudio node to join audio files in an audio folder
Introduced GenerateMusic node for generating music from descriptions using the AudioCraft library
Added XttsNode for text-to-speech conversion using the XTTS library
Implemented AdjustAudioVolumeForFiles node to adjust volume for a list of audio files
The changes also include the addition of LiteLLM nodes for text generation:
Added FuncAnysToText node that runs Python code with multiple input types and outputs a string
Introduced FuncAnysToLatent node for running code that outputs a latent representation
Added FuncAnysToString node for executing code with various inputs and producing a string output
Implemented FuncAnysToList node to run code that takes different input types and returns a list
Added FuncAnysToCond node for running code with multiple inputs and generating a conditioning output
These new nodes expand the capabilities for video and audio processing, allowing more advanced manipulations and generation of video and audio content within the graph.
This commit introduces a series of refactoring and enhancements across multiple modules, improving code readability, consistency, and adding new functionalities.
Highlights:
1. `tests/test_hf_diffusers_nodes.py`:
- Removed unnecessary whitespace.
- Improved variable assignment consistency (e.g., `ret=ret[0]` to `ret = ret[0]`).
- Added a new test case `test_create_unet2d_model` for Unet2d model creation.
2. `server_endpoints.py`:
- Cleaned up commented-out code and added more descriptive comments.
- Refactored the construction of `real_git_path` for better readability and future maintenance.
- Implemented minor formatting improvements for better code structure.
3. `basic.py`:
- Introduced a new class `TupleNode` to handle tuple data types.
- Modified `StrBreakout` to include dynamic typing for list conversion.
- Enhanced code readability with better variable naming and added comments for clarity.
4. `pytorch_nodes.py`:
- Added import statements within functions for `torch`.
- New classes `SaveTorchTensor` and `LoadTorchTensor` added for saving and loading torch tensors.
- `GridSearchTraining` class updated with minor formatting changes.
5. `image.py`:
- In `FloodFillNode`, removed a redundant category assignment.
- Added two new classes: `DecorateHeightMap` and `RandomImageFromFolder`. These classes provide functionalities for decorating height maps with images and selecting random images from a folder, respectively.
6. `hf_diffusers_nodes.py`:
- Fixed formatting in tensor format options and added a new line for clarity.
- New class `CreateUnet2dModel` added for creating a UNet2D model with customizable parameters.
- Introduced `TrainUnet2dModel` for training a UNet2D model with a dataset, including detailed setup for the training process.
7. `config.yaml`:
- Added a new base path for `gitio` to centralize the configuration.
These changes aim to improve the maintainability of the codebase, streamline testing processes, and expand the functionalities within the modules, particularly related to image processing and model training in PyTorch and diffusers.
This commit introduces significant enhancements and new features across multiple files in the project. Major changes include:
1. `pytorch_nodes.py`: Expanded PyTorch node functionalities with additional imports, device compatibility checks, optimizer state updates, and training enhancements. Introduced new methods like `optimizer_to` for transferring optimizer state to a specified device, `reset_optimizer_weights` to clear optimizer states, and `make_image_tensor` for converting outputs to image tensors. Also, improved the training process with device compatibility checks, optimizer updates, and the ability to save and load models during training for efficiency.
2. `server_endpoints.py`: Modified server endpoints for improved git integration and user management. Adjustments include dynamic handling of different git repositories, refined branch and user management in GitIO class, and the integration of user-specific git settings in user management functions.
3. `config.yaml`: Added new configurations for temporary directory and git repositories, providing more flexibility and control over file management and version control integration.
4. `torchvision_nodes.py`: Introduced a new file to integrate torchvision functionalities. It includes classes for transform composition, application of torchvision transforms, and handling of torchvision-specific data structures.
5. `hf_diffusers_nodes.py`: Integrated Hugging Face's diffusers library with new nodes for loading and utilizing DDPMPipelines, handling tensor to image conversions, and managing datasets from the Hugging Face ecosystem.
6. `image.py`, `video.py`, `functional.py`: Refactored and enhanced existing functionalities. Included minor tweaks and bug fixes to improve the overall efficiency and robustness of image and video processing capabilities.
7. `requirements.txt`: Updated dependencies, replacing `fastnoisesimd` with `diffusers` to align with the newly integrated Hugging Face diffusers functionalities.
8. `config.py`: Improved the configuration file loading mechanism with dynamic path handling and the ability to replace placeholders with actual paths, enhancing the flexibility of configuration management.
This comprehensive update significantly enhances the project's capabilities in machine learning, especially in PyTorch and torchvision, and improves server endpoint functionalities for better user and version control management.
Imports Repo from the git module.
Removes unused imports such as InvalidGitRepositoryError, os, copy, re, and imports from custom modules.
Removes the check_for_js_extension() function and its related logic, which checked for and copied JavaScript extension files.
image.py:
Modifies the ImageStackAndMatch class:
Updates the input type specifications.
Adds a new function hex_to_rgb to convert hex color codes to RGB tuples.
Adds a new function pad_to_max_size to pad images to a specified size.
Adds a new function resize_image to resize images with interpolation.
Modifies the process_image function to use the new functions for scaling and padding.
Modifies the process_images function to process up to four images, scaling them, and stacking them into a tensor.
functional.py:
Adds a new class FuncAnysToLatent:
This class runs eval on given text with five "any" inputs and outputs a latent value.
Specifies input types for five "any" inputs and the text to evaluate.
Provides a func function that executes the provided code with the inputs and returns the result as a latent value.
basic.py:
Modifies the get_value function:
Adds a check to ensure that LIST is a list type before returning it; otherwise, it returns an error message.
This file introduces a module containing various PyTorch-related classes and functionalities for use within the Eternal Toolkit (ETK) environment.
It imports essential PyTorch modules such as torch, torch.nn, and others.
Defines two dictionaries, NODE_CLASS_MAPPINGS and NODE_DISPLAY_NAME_MAPPINGS, for mapping class names to display names.
Implements a decorator ETK_pytorch_base to customize attributes of PyTorch-related classes.
Contains multiple PyTorch-related classes with the following key functionalities:
PyTorchDatasetDownloader: Downloads PyTorch datasets and returns them as tensors.
DatsetSplitter: Splits a dataset into training and validation sets.
AddLinearLayerNode: Adds linear layers to PyTorch models.
AddSoftmaxLayerNode: Adds softmax layers to PyTorch models.
SequentialModelProvider: Provides default sequential PyTorch models.
PyTorchInferenceNode: Performs inference using pre-trained PyTorch models.
FlattenTensor and FlattenDataset: Flatten tensors and datasets.
ReshapeTensor: Reshapes tensors.
ChangeTensorType: Changes the data type of tensors.
TensorToList: Converts tensors to lists.
SliceTensor: Slices tensors.
SaveModel and LoadModel: Save and load PyTorch models.
TrainModel: Trains PyTorch models on datasets.
TensorsToDataset and DatasetToDataloader: Convert tensors to datasets and datasets to data loaders, respectively.
Each class has well-defined input types, return types, and functions tailored to specific PyTorch operations.
Exception handling and error messages are provided to ensure robustness and usability.
The classes are designed to work seamlessly with both CPU and GPU devices based on the specified device parameter.
These classes extend the capabilities of ETK by providing powerful PyTorch functionalities for model development and data processing.
These new files and their contents enhance the functionality of ETK by enabling users to work with PyTorch models, datasets, and tensors more effectively. They offer a wide range of operations for deep learning and machine learning tasks within the ETK environment.
This file introduces a Python script for managing JavaScript extension files within a specific directory structure.
It includes an import statement for the os module.
Imports essential variables ETK_PATH and GDE_PATH from the custom_nodes.EternalKernelLiteGraphNodes.local_shared module.
Defines a function called check_for_js_extension(), which:
Verifies the existence of JavaScript extension files.
Compares modification timestamps to determine which file is newer.
Copies files between directories if they are outdated.
Provides user feedback during the copying process.
Calls the check_for_js_extension() function at the end of the script to perform the checks and copying.
Shared model instances across TinyTxtToImg objects for efficiency
Added option to unload models from memory
Improved CUDA and CPU memory handling
Introduced ScaleLatentChannelwise for latent scaling
Added TextInImage and DetectTextLines for OCR capabilities
Removed unnecessary comments and adjusted code formatting
updated openai and openai_function_calling using:
pip install --upgrade {package name}
found a missing RETURN_TYPES = ("IMAGE",) in ETKLGN/image.PreviewImageTest
This commit adds several new nodes to enable flexible typing and conversion between types:
- ValueToAny - Converts primitive values like ints, floats, strings, bools to Any type
- IterableToAny - Converts iterable values like lists, dicts to Any
- CallableToAny - Converts callables like functions to Any
- ObjectToAny - Converts tensor objects like images, latents etc to Any
The motivation is to allow passing different types flexibly into functions using Any typing. This enables writing functions in a more generic way by accepting Any instead of specific types.
The nodes convert the input values to Any and return debug info on what was converted. This allows tracing what types entered a function even if the function uses generic Any types.
No breaking changes, new nodes added:
- ValueToAny
- IterableToAny
- CallableToAny
- ObjectToAny
These provide building blocks for more flexible typing, opening up possibilities like:
- Writing utility and processing functions accepting Any instead of specific types
- Calling functions dynamically with values of different types
- Tracing and logging what types passed through a function
Next steps could include AnyToValue nodes, more specialized Any converters, and exploring uses for generic functions.
started creating whisper transcription node
modified how imports are handled to avoid importing the same moduels twice under different name spaces.
added basic.py to house simple functions that work on primitives.
NOTE: this update relies on change to /web/extensions that are not included in this commit, a future commit will include an install process for these changes.
This commit refactors the handling of the 'steps' attribute, making it configurable via arguments, and adds memory cleanup logic within the 'stackme' method. Additionally, conditional cloning of images ensures that operations are performed only if the images are not None. Commented-out code provides a placeholder for potential future modifications.
In the func method, the variable name 'text' has been updated to 'lst' to better reflect its purpose. Additionally, a new variable 'x_list' has been assigned the value of 'lst', which may be a preparation for further enhancements.
This commit introduces the JoinAudioFiles class within the ETK/audio category. The class provides functionality to concatenate a list of audio files into a single file, using librosa and soundfile libraries. Metadata such as duration and samplerate are also retrieved. The code complies with the preferred import style.
- DecodeTranscriptNode to parse transcript text
- YoutubeUploadNode to upload videos to YouTube
- Audio metadata and file list nodes
- Image stack blending and stripping alpha channel
- Sampler dataset creation node
- Utils for deep copying tensors and arrays
Also includes:
- Testing for new utils module
- Git nodes marked with base class
- Renamed nodes.py to avoid conflicts
- Import refactoring to handle naming
Specifics:
- descript.py: New transcript parsing node
- youtube_nodes.py: New YouTube upload node
- audio.py: New audio metadata and file list nodes
- utils.py: Deep copy utility functions
- image.py: New image stack blending and alpha stripping
- tests/test_utils.py: Tests for new utils
- git.py: Marked git nodes with base class
- __init__.py: Refactored imports and mappings
- requirements.txt: Added YouTube dependencies
- functional.py: Minor tweaks
- modules/youtube.py: Helper for upload node
- video.py: Minor cleanup
- tests/test_youtube_nodes.py: Test for YouTube upload
- Introduced widget definitions for Git-related operations in `bases.py`.
These changes lay the foundation for Git-related functionality in the codebase and provide widget definitions for interacting with Git repositories.
Tasha Upchurch