nodes/HowToSD-ComfyUI-Data-Analysis
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Added Pandas Melt node.

Browse Source
This commit is contained in:
HowToSD
2025-02-23 13:22:12 -08:00
parent f4e1b843a8
commit b9f1c7f89e
11 changed files with 483 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
__init__.py
+3
View File
@@ -60,6 +60,7 @@ from .modules.pandas_wrapper.pandas_log import PandasLog
from .modules.pandas_wrapper.pandas_max import PandasMax
from .modules.pandas_wrapper.pandas_mean import PandasMean
from .modules.pandas_wrapper.pandas_median import PandasMedian
from .modules.pandas_wrapper.pandas_melt import PandasMelt
from .modules.pandas_wrapper.pandas_min import PandasMin
from .modules.pandas_wrapper.pandas_mode import PandasMode
from .modules.pandas_wrapper.pandas_mul import PandasMul
@@ -172,6 +173,7 @@ NODE_CLASS_MAPPINGS: Dict[str, Type[T]] = {
"PandasMax": PandasMax,
"PandasMean": PandasMean,
"PandasMedian": PandasMedian,
"PandasMelt": PandasMelt,
"PandasMin": PandasMin,
"PandasMode": PandasMode,
"PandasMul": PandasMul,
@@ -284,6 +286,7 @@ NODE_DISPLAY_NAME_MAPPINGS: Dict[str, str] = {
"PandasMax": "Pandas Max",
"PandasMean": "Pandas Mean",
"PandasMedian": "Pandas Median",
"PandasMelt": "Pandas Melt",
"PandasMin": "Pandas Min",
"PandasMode": "Pandas Mode",
"PandasMul": "Pandas Mul",
docs/images/melt.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 158 KiB

docs/reference/node_reference.md
+1
View File
@@ -136,6 +136,7 @@
| [Pandas Horizontal Concat](pandas_horizontal_concat.md) | Horizontally concatenates two pandas DataFrames. |
| [Pandas Horizontal Split](pandas_horizontal_split.md) | Horizontally splits a Pandas DataFrame into two pandas DataFrames. |
| [Pandas Join](pandas_join.md) | Merges two pandas DataFrames based on a common column. |
| [Pandas Melt](pandas_melt.md) | Performs an unpivot operation (melt) on a pandas DataFrame. |
| [Pandas Pivot](pandas_pivot.md) | Computes a pivot table from a pandas DataFrame. |
| [Pandas Rename](pandas_rename.md) | Renames an index (row label) or a column label. |
| [Pandas Series To Data Frame](pandas_series_to_dataframe.md) | Converts a Pandas Series to a DataFrame. |
docs/reference/pandas_melt.md
+109
View File
@@ -0,0 +1,109 @@
# Pandas Melt
Performs an unpivot operation (melt) on a pandas DataFrame.
## Understanding Melt (Unpivot)
The melt operation converts a DataFrame from **wide format** (where multiple columns represent separate variables) into **long format** (where values are stacked under a single column).
### Example: Frozen Food Prices Table
Given the following DataFrame:
```
Frozen Food Prices
Manufacturer Pizza Burger
---------------------------------------------------
Aimee Italian Foods 10 12
Olivia Cuisine 11 8
Maddie Foods 12 7
```
To demonstrate **melting a single column**, we will **only transform the `"Pizza"` column** and **temporarily exclude `"Burger"`** from the output. `"Burger"` is not affected in this step because we are only selecting `"Pizza"` for melting.
```
Frozen Food Prices
Manufacturer X Y
---------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
```
Here:
- Column `X` is assigned to `"Item"` (`var_name`).
- Column `Y` is assigned to `"Price"` (`value_name`).
- `"Pizza"` is explicitly specified in `value_vars` as the column being unpivoted.
- `"Manufacturer"` remains unchanged and is included in `id_vars` to preserve row identity.
The final transformed DataFrame:
```
Frozen Food Prices
Manufacturer Item Price
----------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
```
### Expanding the Melt Operation
If we want to **melt both `"Pizza"` and `"Burger"` together**, we specify both columns in `value_vars`:
```
Frozen Food Prices
Manufacturer Item Price
----------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
Aimee Italian Foods Burger 12
Olivia Cuisine Burger 8
Maddie Foods Burger 7
```
Here, we explicitly define:
- `"Pizza"` and `"Burger"` in `value_vars`.
- A new `"Item"` column (`var_name`), which holds the original column names (`"Pizza"`, `"Burger"`).
- A `"Price"` column, which stores the corresponding values.
To specify both columns in this node, provide:
```
Pizza,Burger
```
This node automatically converts the string into a list and passes it to `pandas.DataFrame.melt()`.
![Melt example](../images/melt.png)
Refer to the [Pandas documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.melt.html) for more information.
## Limitation
MultiIndex columns are not supported.
## Input
| Name | Data type |
|---|---|
| dataframe | DataFrame |
| id_vars | String |
| value_vars | String |
| var_name | String |
| value_name | String |
| ignore_index | Boolean |
## Output
| Data type |
|---|
| DataFrame |
<HR>
Category: Transformation
ComfyUI Data Analysis Node Reference. © 2025 Hide Inada (HowToSD.com). All rights reserved.
examples/workflows/melt.json
+1
View File
@@ -0,0 +1 @@
{"last_node_id":17,"last_link_id":14,"nodes":[{"id":12,"type":"PandasShowDataFrame","pos":[-1679.7327880859375,6962.63037109375],"size":[329.3037414550781,107.76365661621094],"flags":{},"order":1,"mode":0,"inputs":[{"name":"dataframe","type":"DATAFRAME","link":10}],"outputs":[{"name":"STRING","type":"STRING","links":null,"shape":6}],"properties":{"Node name for S&R":"PandasShowDataFrame"},"widgets_values":[" Manufacturer Pizza Burger\n0 Aimee Italian Foods 10 12\n1 Olivia Cuisine 11 8\n2 Maddie Foods 12 7"]},{"id":14,"type":"PandasShowDataFrame","pos":[-1328.8665771484375,7141.0478515625],"size":[284.7939758300781,116.40038299560547],"flags":{},"order":4,"mode":0,"inputs":[{"name":"dataframe","type":"DATAFRAME","link":12}],"outputs":[{"name":"STRING","type":"STRING","links":null,"shape":6}],"properties":{"Node name for S&R":"PandasShowDataFrame"},"widgets_values":[" Manufacturer Item Price\n0 Aimee Italian Foods Pizza 10\n1 Olivia Cuisine Pizza 11\n2 Maddie Foods Pizza 12"]},{"id":13,"type":"PandasMelt","pos":[-1679.666259765625,7141.65576171875],"size":[315,178],"flags":{},"order":2,"mode":0,"inputs":[{"name":"dataframe","type":"DATAFRAME","link":11}],"outputs":[{"name":"DATAFRAME","type":"DATAFRAME","links":[12],"slot_index":0}],"properties":{"Node name for S&R":"PandasMelt"},"widgets_values":["Manufacturer","Pizza","Item","Price",0]},{"id":17,"type":"PandasMelt","pos":[-1683.4326171875,7385.7001953125],"size":[315,178],"flags":{},"order":3,"mode":0,"inputs":[{"name":"dataframe","type":"DATAFRAME","link":13}],"outputs":[{"name":"DATAFRAME","type":"DATAFRAME","links":[14],"slot_index":0}],"properties":{"Node name for S&R":"PandasMelt"},"widgets_values":["Manufacturer","Pizza,Burger","Item","Price",true]},{"id":15,"type":"PandasShowDataFrame","pos":[-1331.198974609375,7390.00537109375],"size":[294.8554382324219,158.5748748779297],"flags":{},"order":5,"mode":0,"inputs":[{"name":"dataframe","type":"DATAFRAME","link":14}],"outputs":[{"name":"STRING","type":"STRING","links":null,"shape":6}],"properties":{"Node name for S&R":"PandasShowDataFrame"},"widgets_values":[" Manufacturer Item Price\n0 Aimee Italian Foods Pizza 10\n1 Olivia Cuisine Pizza 11\n2 Maddie Foods Pizza 12\n3 Aimee Italian Foods Burger 12\n4 Olivia Cuisine Burger 8\n5 Maddie Foods Burger 7"]},{"id":10,"type":"PandasCreate","pos":[-1952.2723388671875,6961.17138671875],"size":[210,120.59086608886719],"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"DATAFRAME","type":"DATAFRAME","links":[10,11,13],"slot_index":0}],"properties":{"Node name for S&R":"PandasCreate"},"widgets_values":["Manufacturer,Pizza,Burger\nAimee Italian Foods,10,12\nOlivia Cuisine,11,8\nMaddie Foods,12,7\n"]}],"links":[[10,10,0,12,0,"DATAFRAME"],[11,10,0,13,0,"DATAFRAME"],[12,13,0,14,0,"DATAFRAME"],[13,10,0,17,0,"DATAFRAME"],[14,17,0,15,0,"DATAFRAME"]],"groups":[],"config":{},"extra":{"ds":{"scale":0.0860798070987705,"offset":[8287.305591731167,3463.780243294214]},"node_versions":{"data-analysis":"94680d227e6b18b5992578b3c77f471b3c19666b"},"ue_links":[]},"version":0.4}
modules/pandas_wrapper/pandas_melt.py
+168
View File
@@ -0,0 +1,168 @@
from typing import Any, Dict
import pandas as pd
from .utils import column_labels_string_to_list
class PandasMelt:
"""
Performs an unpivot operation (melt) on a pandas DataFrame.
## Understanding Melt (Unpivot)
The melt operation converts a DataFrame from **wide format** (where multiple columns represent separate variables) into **long format** (where values are stacked under a single column).
### Example: Frozen Food Prices Table
Given the following DataFrame:
```
Frozen Food Prices
Manufacturer Pizza Burger
---------------------------------------------------
Aimee Italian Foods 10 12
Olivia Cuisine 11 8
Maddie Foods 12 7
```
To demonstrate **melting a single column**, we will **only transform the `"Pizza"` column** and **temporarily exclude `"Burger"`** from the output. `"Burger"` is not affected in this step because we are only selecting `"Pizza"` for melting.
```
Frozen Food Prices
Manufacturer X Y
---------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
```
Here:
- Column `X` is assigned to `"Item"` (`var_name`).
- Column `Y` is assigned to `"Price"` (`value_name`).
- `"Pizza"` is explicitly specified in `value_vars` as the column being unpivoted.
- `"Manufacturer"` remains unchanged and is included in `id_vars` to preserve row identity.
The final transformed DataFrame:
```
Frozen Food Prices
Manufacturer Item Price
----------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
```
### Expanding the Melt Operation
If we want to **melt both `"Pizza"` and `"Burger"` together**, we specify both columns in `value_vars`:
```
Frozen Food Prices
Manufacturer Item Price
----------------------------------------------------
Aimee Italian Foods Pizza 10
Olivia Cuisine Pizza 11
Maddie Foods Pizza 12
Aimee Italian Foods Burger 12
Olivia Cuisine Burger 8
Maddie Foods Burger 7
```
Here, we explicitly define:
- `"Pizza"` and `"Burger"` in `value_vars`.
- A new `"Item"` column (`var_name`), which holds the original column names (`"Pizza"`, `"Burger"`).
- A `"Price"` column, which stores the corresponding values.
To specify both columns in this node, provide:
```
Pizza,Burger
```
This node automatically converts the string into a list and passes it to `pandas.DataFrame.melt()`.
![Melt example](../images/melt.png)
Refer to the [Pandas documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.melt.html) for more information.
## Limitation
MultiIndex columns are not supported.
category: Transformation
"""
@classmethod
def INPUT_TYPES(cls) -> Dict[str, Any]:
"""
Defines the input types for the `select_columns` function.
Returns:
Dict[str, Any]: A dictionary specifying required input types.
"""
return {
"required": {
"dataframe": ("DATAFRAME", {}),
"id_vars": ("STRING", {"default": ""}),
"value_vars": ("STRING", {"default": ""}),
"var_name": ("STRING", {"default": ""}),
"value_name": ("STRING", {"default": ""}),
"ignore_index": ("BOOLEAN", {"default": True})
}
}
RETURN_TYPES: tuple = ("DATAFRAME",)
FUNCTION: str = "f"
CATEGORY: str = "Data Analysis"
def f(self, dataframe: pd.DataFrame,
id_vars: str,
value_vars: str,
var_name: str,
value_name: str,
ignore_index: bool) -> tuple:
"""
Selects specific columns from a pandas DataFrame.
Args:
dataframe (DataFrame): The input DataFrame.
id_vars (str): Comma-separated string to specify ID variable columns.
value_vars (str): Comma-separated string to specify column(s) to be unpivoted.
var_name (str): New column label for the column that contains original column labels as row values.
value_name (str): Name of the new column that holds the values for columns specified in `value_vars`.
ignore_index (bool): Flag that specifies whether to keep the original index.
If ``True``, a new sequential index is generated.
If ``False``, the original index is retained and repeated as needed.
Returns:
tuple: A tuple containing the DataFrame.
"""
id_vars = column_labels_string_to_list(dataframe, id_vars)
value_vars = column_labels_string_to_list(dataframe, value_vars)
var_name = column_labels_string_to_list(dataframe, var_name, no_check=True)
value_name = column_labels_string_to_list(dataframe, value_name, no_check=True)
if len(id_vars) == 1:
id_vars = id_vars[0]
if len(value_vars) == 1:
value_vars = value_vars[0]
if len(var_name) == 0:
var_name = None
elif len(var_name) == 1:
var_name = var_name[0]
if len(value_name) == 0:
value_name = None
elif len(value_name) == 1:
value_name = value_name[0]
df2 = dataframe.melt(
id_vars=id_vars,
value_vars=value_vars,
var_name=var_name,
value_name=value_name,
ignore_index=ignore_index)
return (df2,)
modules/pandas_wrapper/utils.py
+31 -5
View File
@@ -1,11 +1,34 @@
from typing import List, Union
from typing import Any, List, Tuple, Union, Set
import json
import numpy as np
import pandas as pd
import pandas as pd
def column_labels_string_to_list(dataframe: pd.DataFrame, s: str) -> List[Union[str, int]]:
def flatten_list_tuple_unique(lst: Union[List[Any], Tuple[Any, ...]]) -> Set[Any]:
"""
Flattens a nested list or tuple and returns a set of unique elements.
Args:
lst (Union[List[Any], Tuple[Any, ...]]):
A list or tuple containing elements of any type, including nested lists/tuples.
Returns:
Set[Any]: A set containing unique elements from the input, flattened.
"""
result: Set[Any] = set()
def recurse(sublist: Union[List[Any], Tuple[Any, ...]]) -> None:
for item in sublist:
if isinstance(item, (list, tuple)):
recurse(item)
else:
result.add(item)
recurse(lst)
return result
def column_labels_string_to_list(dataframe:pd.DataFrame, s:str, no_check:bool=False) -> List[Union[str, int]]:
"""
Converts a comma-separated string of column labels into a list of valid column names for a Pandas DataFrame.
@@ -16,6 +39,8 @@ def column_labels_string_to_list(dataframe: pd.DataFrame, s: str) -> List[Union[
Args:
dataframe (pd.DataFrame): The DataFrame to validate column names against.
s (str): A comma-separated string representing column labels.
no_check (str): If True, input string is processed as a string without checking the column labels
of the DataFrame.
Returns:
List[Union[str, int]]: A list of valid column labels.
@@ -24,11 +49,12 @@ def column_labels_string_to_list(dataframe: pd.DataFrame, s: str) -> List[Union[
ValueError: If any label in `s` does not match an existing column name.
"""
columns = [col.strip() for col in s.split(",")]
valid_columns = set(dataframe.columns) # Use a set for faster lookups
valid_columns = flatten_list_tuple_unique(dataframe.columns)
output_columns = []
for c in columns:
if c in valid_columns:
if c in valid_columns or no_check:
output_columns.append(c)
else:
try:
pyproject.toml
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "data-analysis"
description = "Data analysis custom modules for ComfyUI - Use Pandas & Matplotlib within ComfyUI"
version = "1.1.23"
version = "1.1.24"
license = {file = "LICENSE"}
[project.urls]
tests/pandas_wrapper/pandas_melt_test.py
+125
View File
@@ -0,0 +1,125 @@
import os
import sys
import unittest
import pandas as pd
PROJECT_ROOT = os.path.realpath(os.path.join(__file__, "..", "..", ".."))
MODULE_ROOT = os.path.join(PROJECT_ROOT, "modules")
sys.path.append(MODULE_ROOT)
from pandas_wrapper.pandas_melt import PandasMelt
class TestPandasMelt(unittest.TestCase):
def setUp(self):
"""Set up test data"""
self.node = PandasMelt()
self.df = pd.DataFrame({
"Manufacturer": ["Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods"],
"Pizza": [10, 11, 12],
"Burger": [12, 8, 7]
})
def test_pandas_melt_single_column(self):
"""Test melting a single column"""
result_df = self.node.f(
dataframe=self.df,
id_vars="Manufacturer",
value_vars="Pizza",
var_name="Item",
value_name="Price",
ignore_index=True
)[0] # Extract DataFrame
expected_df = pd.DataFrame({
"Manufacturer": ["Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods"],
"Item": ["Pizza", "Pizza", "Pizza"],
"Price": [10, 11, 12]
})
pd.testing.assert_frame_equal(result_df, expected_df)
def test_pandas_melt_multiple_columns(self):
"""Test melting multiple columns"""
result_df = self.node.f(
dataframe=self.df,
id_vars="Manufacturer",
value_vars="Pizza,Burger",
var_name="Item",
value_name="Price",
ignore_index=True
)[0] # Extract DataFrame
expected_df = pd.DataFrame({
"Manufacturer": [
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods",
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods"
],
"Item": ["Pizza", "Pizza", "Pizza", "Burger", "Burger", "Burger"],
"Price": [10, 11, 12, 12, 8, 7]
})
pd.testing.assert_frame_equal(result_df, expected_df)
def test_pandas_melt_ignore_index_true(self):
"""Test melting with ignore_index=True"""
# Modify the original DataFrame to have a custom index
self.df.index = pd.Index([10, 20, 30]) # Explicitly set a non-default index
result_df = self.node.f(
dataframe=self.df,
id_vars="Manufacturer",
value_vars="Pizza,Burger",
var_name="Item",
value_name="Price",
ignore_index=True
)[0] # Extract DataFrame
expected_df = pd.DataFrame({
"Manufacturer": [
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods",
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods"
],
"Item": ["Pizza", "Pizza", "Pizza", "Burger", "Burger", "Burger"],
"Price": [10, 11, 12, 12, 8, 7]
})
# Check the data itself
pd.testing.assert_frame_equal(result_df, expected_df, check_index_type=False)
# Explicitly verify that the index is reset to a sequential range
expected_index = pd.RangeIndex(start=0, stop=len(expected_df), step=1)
self.assertTrue(result_df.index.equals(expected_index), "Index is not fully reset.")
def test_pandas_melt_ignore_index_false(self):
"""Test melting with ignore_index=False"""
self.df.index = pd.Index([10, 20, 30]) # Explicitly set a non-default index
result_df = self.node.f(
dataframe=self.df,
id_vars="Manufacturer",
value_vars="Pizza,Burger",
var_name="Item",
value_name="Price",
ignore_index=False
)[0] # Extract DataFrame
expected_df = pd.DataFrame({
"Manufacturer": [
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods",
"Aimee Italian Foods", "Olivia Cuisine", "Maddie Foods"
],
"Item": ["Pizza", "Pizza", "Pizza", "Burger", "Burger", "Burger"],
"Price": [10, 11, 12, 12, 8, 7]
}, index=[10, 20, 30, 10, 20, 30]) # Correctly retain and repeat original index
# Compare both data and index
pd.testing.assert_frame_equal(result_df, expected_df)
# Explicitly verify that the index is correctly retained and repeated
expected_index = pd.Index([10, 20, 30, 10, 20, 30])
self.assertTrue(result_df.index.equals(expected_index), "Index was not retained correctly.")
if __name__ == "__main__":
unittest.main()
tests/pandas_wrapper/utils_column_labels_string_to_list_test.py
+8
View File
@@ -21,6 +21,8 @@ class TestColumnLabelsStringToList(unittest.TestCase):
20: [13, 14, 15] # Another integer column name
})
self.no_check_labels = ["X", "Y"]
def test_valid_column_strings(self):
"""Test valid column name conversion from string to list."""
self.assertEqual(column_labels_string_to_list(self.df, "A,B"), ["A", "B"])
@@ -60,6 +62,12 @@ class TestColumnLabelsStringToList(unittest.TestCase):
column_labels_string_to_list(self.df, "A,xyz")
self.assertIn("Column 'xyz' not found in the DataFrame.", str(context.exception))
def test_no_check_labels(self):
"""Test no check labels"""
self.assertEquals(
column_labels_string_to_list(self.df, "X,Y", no_check=True),
self.no_check_labels)
if __name__ == "__main__":
unittest.main()
tests/pandas_wrapper/utils_flatten_list_tuple_unique_test.py
+36
View File
@@ -0,0 +1,36 @@
import os
import sys
import unittest
from typing import Any, List, Tuple, Union, Set
PROJECT_ROOT = os.path.realpath(os.path.join(__file__, "..", "..", ".."))
MODULE_ROOT = os.path.join(PROJECT_ROOT, "modules")
sys.path.append(MODULE_ROOT)
from pandas_wrapper.utils import flatten_list_tuple_unique
class TestFlattenListTupleUnique(unittest.TestCase):
def setUp(self):
self.test_cases = [
(["a", "b", "c"], {"a", "b", "c"}), # Strings only
([1, 2, 3], {1, 2, 3}), # Integers only
(["a", 1], {"a", 1}), # Mixed types
(["a", ("b", "c", "d"), (1, 2, 3), ("b", "c", "e")], {"a", "b", "c", "d", 1, 2, 3, "e"}), # Nested tuples
([("x", "y"), ["y", "z"]], {"x", "y", "z"}), # Nested lists and tuples
([1, (2, [3, (4, 5)]), 6, (7, [8, 9])], {1, 2, 3, 4, 5, 6, 7, 8, 9}), # Deep nesting
([], set()), # Empty list
([()], set()), # Empty tuple inside a list
([None, [None, None]], {None}), # None values
([True, False, 1, 0], {True, False}), # Boolean values (Python treats True as 1 and False as 0)
]
def test_flatten_list_tuple_unique(self):
for input_data, expected_output in self.test_cases:
with self.subTest(input_data=input_data):
self.assertEqual(flatten_list_tuple_unique(input_data), expected_output)
if __name__ == "__main__":
unittest.main()