Audio Loader: For loading an audio file using the librosa library.
Audio Feature Calculator: For calculating the audio feature modifiers for each step in the given audio.
Prompt Sequence Builder: For stacking multiple ComfyUI prompts into a Prompt Sequence.
Prompt Sequence Interpolator: For calculating additional sub-prompts within each set of prompts in Prompt Sequence to create a weighted interpolation transition between each.
Prompt Sequence Renderer: For rendering a sequence of images with a variety of parameters from a Prompt Sequence.
Image Concatenator For combining multiple images into a single Tensor. Images are not visually combined and are still distinct.

Types

AUDIO (tuple[ndarray, float]): For representing loaded audio files.
TENSOR_1D (Tensor): For representing a 1D Tensor of data.
PROMPT_SEQ (list[MbmPrompt]): For representing multiple prompts (positive and negative) in an ordered sequence.

Install

Enter ComfyUI's Python Environment by running .\.venv\Scripts\activate from ComfyUI's root directory.
Clone this repo into ComfyUI's custom_nodes directory by entering the directory and running: git clone <LINK_TBD> MBM_MusicVisualizer.
Enter the MBM_MusicVisualizer directory.
Run pip install -r .\requirements.txt to install this project's dependencies.
Start ComfyUI as normal.

Nodes will be found in the MBMnodes/ submenu inside ComfyUI.

Usage

📝 Note: Drag the above image (or the example workflow file) into ComfyUI to automatically load this flow!

Place any audio files you would like to load in the audio/ directory. You can always refresh the webpage ComfyUI is loaded into to refresh the list in the Audio Loader node.

The Music Visualizer node takes an AUDIO object in and produces a set of Latent Images on output. A tqdm progress bar will be shown in the console to display the current status of the visualization and how long it is expected to take.

Upon completion of a visualization, the Music Visualizer will output the input FPS, a set of Latent Images that can be decoded using the built-in Latent Decoder, and a set of Images showing relevant data from the run. The FPS can be fed into any further video or gif generating nodes. The Latent Images, which are the output content of the visualization, should be converted and either saved individually or compiled into a video through another node. The charts are pixel images that can be saved or modified as desired.

When testing your generations, consider setting the image_limit to 1 or higher to generate only a specific number of images. Doing so will still produce complete charts for most data sources allowing you to preview the general flow of the visualization before you commit to image generation for all frames.