Update pyproject.toml

- Improve status message.
- Enhance node flexibility.
- Optimize frame processing.

README.md

@@ -1,21 +1,9 @@
 # ComfyUI_FlipStreamViewer
 
-**ComfyUI_FlipStreamViewer** is a tool that provides a customizable viewer interface for flipping images with frame interpolation, allowing you to watch high-fidelity pseudo-videos without needing AnimateDiff.
+**ComfyUI_FlipStreamViewer** is a tool that provides a customizable viewer interface for flipping images with frame interpolation.
 
 <div><video controls src="https://github.com/user-attachments/assets/2a40c7c6-045e-4d86-a0fd-51a24f5472b4"></video></div>
 
-## Required Custom Nodes
-
-Please install the following nodes via ComfyUI-Manager:
-
-- ComfyUI Impact Pack  ; Required
-- LoRA Tag Loader for ComfyUI  ; Optional for workflow
-- ComfyUI-Frame-Interpolation  ; Optional for FlipStreamFilmVfi
-- ComfyUI-WD14-Tagger  ; Optional for Tagger
-- ComfyUI-Florence2  ; Optional for FlipStreamSegMask
-- ComfyUI_TensorRT  ; Optional for FlipStreamFileSelect_TensorRT
-- ComfyUI-AnimateDiff-Evolved  ; Optional for FlipStreamFileSelect_AnimateDiffModel
-
 ## Getting Started
 1. **Run ComfyUI** and from ComfyUI-Manager, select Manager -> Custom Node Manager and install ComfyUI_FlipStreamViewer.
 2. **Restart ComfyUI** and load the `simple.json` file located in the `workflows` folder.
@@ -31,28 +19,47 @@ Please install the following nodes via ComfyUI-Manager:
 
 6. The 'preset' folder may be created in the current directory for saving and loading preset files.
 
-### Limitations
+## Limitations
 
 - Please note that this tool is not designed with security measures against third-party attacks. Therefore, it should only be used for personal purposes within a secure network.
 - There is no save video function available; please customize your workflow as needed.
 
+## Optional Custom Nodes dependencies
+
+The following custom nodes can be used as needed:
+- ComfyUI-WD14-Tagger: For Tagger
+- ComfyUI-Inspyrenet-Rembg: For FlipStreamRembg
+- ComfyUI-Florence2: For FlipStreamSegMask
+- ComfyUI-Frame-Interpolation: For FlipStreamFilmVfi
+
+The following custom nodes may also be used within workflows:
+- ComfyUI Impact Pack
+- LoRA Tag Loader for ComfyUI
+- ComfyUI-AnimateDiff-Evolved
+- ComfyUI-VideoHelperSuite
+- ComfyUI-DepthAnythingV2
+
 ## UI Features
 
-- **Customizable Left Panel**: The left panel UI is customizable with UI nodes such as FlipStreamSlider, FlipStreamTextBox, FlipStreamInputBox, FlipStreamSelectBox_*, FlipStreamFileSelect_*, and FlipStreamPreviewBox.
+- **Customizable Left Panel**: The left panel UI is customizable with UI Nodes.
 - **UI Node Ordering**: These UI nodes are ordered by their titles, and you can change the node titles to something like '10.steps' or '11.cfg'. You can also click on the title to compact the node on the workflow.
-- **Label Input**: Each UI node has a 'label' input, which should be a unique identifier used as the parameter name.
+- **Label Input**: Each UI node has a 'label' input, which should be a unique identifier used as the parameter name. 'loramode', 'lora', 'offsetX', 'offsetY' and 'scale' are internally used, so they cannot be used for labels.
 - **Enable Output**: Each UI node also has an 'enable' output to use with 'Control Bridge' for switching to bypass some nodes or FlipStreamSwitch* for switching input.
 - **Right Panel**: You can operate several prepared functions such as Status, Darker, Tagger, Preset, and Lora.
 - **Status**: The right panel shows ComfyUI status and error information.
 - **Darker**: You can adjust the image brightness on the viewer using the darker parameter. It can also be set via query:
 `http://localhost:8188/flipstreamviewer?darker=0.33`
 - **Tagger**: You can capture screenshots and generate tags from images using WD14. It depends on ComfyUI-WD14-Tagger.
-- **Preset**: You can save and load parameters that are set on viewer controls. The 'M' button means Move to another folder.
+- **Preset**: You can save and load parameters that are set on viewer controls. The 'M' button means Move to another folder. It can also be set via query:
+`http://localhost:8188/flipstreamviewer?showPresetDialog=1&presetFolder=folder_name&presetFile=file_name.json`
 - **Lora**: You can select LoRA and choose tags. It can also choose random tags. The LoRA preview box can be clicked to jump to the Civitai LoRA page if found. The 'M' button means Move to another folder. The 'T' button means Toggle. The 'R' button means Random choose.
+- **Toggle View**: You can click the center panel to hide the left and right panels, and a subsequent click will hide the image. It can also be set via query: 
+`http://localhost:8188/flipstreamviewer?toggleView=1`
 - **Auto Hide**: Stream and preview in the viewer will automatically hide after 5 minutes if the page is not reloaded.
 
-## Nodes
+## UI Nodes
 
+- **FlipStreamSection**: A section label for UI Nodes.
 - **FlipStreamSlider**: A slider for adjusting values.
 - **FlipStreamTextBox**: A text box for inputting multiline text.
 - **FlipStreamInputBox**: An input box for various boxtype inputs. The 'U' button means update. You can choose special boxtype 'seed' or 'r4d' with the 'R' button, which means randomize. The boxtype 'r4d' can be used to generate a 4-digit part of a prompt like 'MOV_{num}'. In this case, FlipStreamTextReplace can help find '{num}' and replace with the output of the input box.
@@ -60,17 +67,24 @@ Please install the following nodes via ComfyUI-Manager:
 - **FlipStreamSelectBox_Scheduler**: A select box for choosing schedulers.
 - **FlipStreamFileSelect_Checkpoints**: A file selector for checkpoints. 'mode' is used to select a subfolder such as 'sd15', 'sdxl', 'pony', or 'flux' in 'checkpoints'. If 'use_lora' is true, it will also affect the 'mode' for the Lora selector. 'use_sub' means using subfolders in the 'mode' folder. 'use_move' means using the move file selector.
 - **FlipStreamFileSelect_VAE**: A file selector for VAE models.
+- **FlipStreamFileSelect_ControlNetModel**: A file selector for ControlNet models.
 - **FlipStreamFileSelect_TensorRT**: A file selector for TensorRT models. It may be used with ComfyUI_TensorRT.
 - **FlipStreamFileSelect_AnimateDiffModel**: A file selector for AnimateDiff models. It may be used with ComfyUI-AnimateDiff-Evolved.
-- **FlipStreamFileSelect_VideoSrc**: A file selector for video sources.
+- **FlipStreamFileSelect_Input**: A file selector for ComfyUI input folder.
+- **FlipStreamFileSelect_Output**: A file selector for ComfyUI output folder.
 - **FlipStreamPreviewBox**: A box for previewing input image.
+
+## Other Nodes
+
 - **FlipStreamSetParam**: A node for setting parameters. However, it needs a reload to update the value in the viewer UI.
 - **FlipStreamGetParam**: A node for getting parameters. 'lora' is a prepared parameter that contains text at the Lora input. Some prompts can contain '{lora}' and FlipStreamTextReplace can help find '{lora}' and replace it with the output of the get param node of 'lora'. 'b64dec' is true for 'lora' and other 'FlipStreamTextBox' parameters, because these multiline parameters are internally base64 encoded.
 - **FlipStreamImageSize**: A node for getting image size.
 - **FlipStreamTextReplace**: A node for replacing text. It will output the result of `text.replace(find, replace.format(value))`.
-- **FlipStreamVideoInput**: A node for video input. FlipStreamFileSelect_VideoSrc can be used to select the path for this node.
+- **FlipStreamScreenGrabber**: A node for grab multiframe screenshots.
+- **FlipStreamSource**: A node for prepare image or latent source.
 - **FlipStreamSwitchImage**: A node for switching images.
 - **FlipStreamSwitchLatent**: A node for switching latents.
+- **FlipStreamRembg**: A node for remove background. It depends on ComfyUI-Inspyrenet-Rembg.
 - **FlipStreamSegMask**: A node for segmentation masks. The target can contain multiple words separated by commas for segmentation. It will use the 'microsoft/Florence-2-large' model, which you can download using the DownloadAndLoadFlorence2Model node of ComfyUI-Florence2. Segmentation sometimes fails, so you may need to try some other random seeds.
 - **FlipStreamBatchPrompt**: A node for simple batch prompting. Use the following format for the input prompt of this node. The 'pre text' and 'append text' sections apply to all frames. The separator `----` should consist of four hyphens, and each line of 'frame text' applies evenly to the number of frames specified in 'frames'. For example, if 'frames' is 8 and there are 2 'frame text' lines, they will be applied starting from frames 0 and 4.
 ```
@@ -97,5 +111,8 @@ append text
 ## Workflow Examples
 
 - **simple.json**: Customized a ComfyUI default workflow for FlipStreamViewer UI nodes.
-- **dmd2_lora.json**: Using DMD2 and some LoRA, only needs 5 steps to generate an image. This workflow depends on 'LoRA Tag Loader for ComfyUI'.
-- **segmask.json**: A 2-step process for generating frames using batch prompts and segmentation mask, followed by interpolation. This workflow depends on 'LoRA Tag Loader for ComfyUI', 'ComfyUI-Frame-Interpolation' and 'ComfyUI-Florence2'. It will use the 'microsoft/Florence-2-large' model, which you can download using the DownloadAndLoadFlorence2Model node of ComfyUI-Florence2, or if you want to skip it, bypass 'FlipStreamSegMask' and 'Set Latent Noise Mask', and it will ignore segmentation mask.
+- **dmd2_lora.json**: Using DMD2 and some LoRA, only needs 5 steps to generate an image.
+- **segmask.json**: A 2-step process for generating frames using batch prompts and segmentation mask, followed by interpolation.
+- **animate.json**: Using AnimateDiff with FlipStreamSource, FlipStreamBatchPrompt, and FlipStreamFilmVfi.
+- **grab_rembg_depth_animate.json**: Using ControlNet and AnimateDiff with FlipStreamScreenGrabber, FlipStreamRembg.
+- **quick_vid2vid.json**: Quick tuning workflow for vid2vid.

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "comfyui_flipstreamviewer"
-description = "ComfyUI_FlipStreamViewer is a tool that provides a customizable viewer interface for flipping images with frame interpolation, allowing you to watch high-fidelity pseudo-videos without needing AnimateDiff."
-version = "3.0.0"
+description = "ComfyUI_FlipStreamViewer is a tool that provides a customizable viewer interface for flipping images with frame interpolation."
+version = "3.1.4"
 license = {file = "LICENSE.txt"}
 
 [project.urls]

requirements.txt

@@ -1 +1,2 @@
-opencv-python
+imageio
+mss

workflows/dmd2_and_lora.json

@@ -480,7 +480,7 @@
       ],
       "size": [
         210,
-        246
+        266
       ],
       "flags": {
         "collapsed": true
@@ -490,13 +490,18 @@
       "inputs": [],
       "outputs": [
         {
-          "name": "path",
+          "name": "file",
           "type": "*",
           "links": [
             10
           ],
           "slot_index": 0
         },
+        {
+          "name": "path",
+          "type": "*",
+          "links": null
+        },
         {
           "name": "enable",
           "type": "BOOLEAN",
@@ -1119,12 +1124,12 @@
       "id": 30,
       "type": "Note",
       "pos": [
-        -239.78614807128906,
-        455.0487976074219
+        -227.77297973632812,
+        -2.78507137298584
       ],
       "size": [
-        275.2828369140625,
-        121.84362030029297
+        370.0531005859375,
+        152.54388427734375
       ],
       "flags": {},
       "order": 12,
@@ -1133,7 +1138,7 @@
       "outputs": [],
       "properties": {},
       "widgets_values": [
-        "Using lora files:\n\ndmd2_sdxl_4step_lora_fp16.safetensors from https://huggingface.co/tianweiy/DMD2/tree/main\n\npixel-art-xl-v1.1.safetensors from\nhttps://civitai.com/models/120096/pixel-art-xl"
+        "This workflow depends on additional custom nodes:\nLoRA Tag Loader for ComfyUI\n\nUsing lora files:\ndmd2_sdxl_4step_lora_fp16.safetensors from https://huggingface.co/tianweiy/DMD2/tree/main\npixel-art-xl-v1.1.safetensors from\nhttps://civitai.com/models/120096/pixel-art-xl"
       ],
       "color": "#432",
       "bgcolor": "#653"
@@ -1369,17 +1374,21 @@
   "config": {},
   "extra": {
     "ds": {
-      "scale": 0.6830134553650707,
+      "scale": 0.5993439296133066,
       "offset": [
-        345.3005491283586,
-        19.586917633514247
+        276.7432726354348,
+        187.22241786354834
       ]
     },
     "node_versions": {
-      "comfyui_flipstreamviewer": "5a8f1af2e82d99fe545dbabbb494647af0b19bbb",
-      "comfy-core": "0.3.13",
+      "ComfyUI_FlipStreamViewer": "c727b903bb52c60932d2cd2b3841c6248d25854c",
+      "comfy-core": "0.3.14",
       "comfyui_lora_tag_loader": "c1e1a061d42384e06ff6c547c4a00aef7cc96e59"
-    }
+    },
+    "VHS_latentpreview": false,
+    "VHS_latentpreviewrate": 0,
+    "VHS_MetadataImage": true,
+    "VHS_KeepIntermediate": true
   },
   "version": 0.4
 }

workflows/segmask.json

@@ -1,5 +1,5 @@
 {
-  "last_node_id": 48,
+  "last_node_id": 49,
   "last_link_id": 89,
   "nodes": [
     {
@@ -60,7 +60,7 @@
       ],
       "size": [
         210,
-        246
+        266
       ],
       "flags": {
         "collapsed": true
@@ -70,13 +70,18 @@
       "inputs": [],
       "outputs": [
         {
-          "name": "path",
+          "name": "file",
           "type": "*",
           "links": [
             10
           ],
           "slot_index": 0
         },
+        {
+          "name": "path",
+          "type": "*",
+          "links": null
+        },
         {
           "name": "enable",
           "type": "BOOLEAN",
@@ -295,45 +300,6 @@
         8
       ]
     },
-    {
-      "id": 33,
-      "type": "FlipStreamFilmVfi",
-      "pos": [
-        2270.01904296875,
-        189.28005981445312
-      ],
-      "size": [
-        210,
-        82
-      ],
-      "flags": {},
-      "order": 29,
-      "mode": 0,
-      "inputs": [
-        {
-          "name": "frames",
-          "type": "IMAGE",
-          "link": 43
-        }
-      ],
-      "outputs": [
-        {
-          "name": "IMAGE",
-          "type": "IMAGE",
-          "links": [
-            44
-          ],
-          "slot_index": 0
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "FlipStreamFilmVfi"
-      },
-      "widgets_values": [
-        8,
-        100
-      ]
-    },
     {
       "id": 5,
       "type": "EmptyLatentImage",
@@ -561,29 +527,6 @@
         "text, watermark"
       ]
     },
-    {
-      "id": 30,
-      "type": "Note",
-      "pos": [
-        -130.1117401123047,
-        429.4935607910156
-      ],
-      "size": [
-        269.9588317871094,
-        93.0940170288086
-      ],
-      "flags": {},
-      "order": 3,
-      "mode": 0,
-      "inputs": [],
-      "outputs": [],
-      "properties": {},
-      "widgets_values": [
-        "Using lora files:\n\ndmd2_sdxl_4step_lora_fp16.safetensors from https://huggingface.co/tianweiy/DMD2/tree/main"
-      ],
-      "color": "#432",
-      "bgcolor": "#653"
-    },
     {
       "id": 15,
       "type": "FlipStreamInputBox",
@@ -598,7 +541,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 4,
+      "order": 3,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -712,7 +655,7 @@
       },
       "widgets_values": [
         "disable",
-        455097485017673,
+        1001307473401221,
         "randomize",
         20,
         1,
@@ -1009,7 +952,7 @@
       },
       "widgets_values": [
         "enable",
-        108965166715117,
+        1100646290435457,
         "randomize",
         20,
         1,
@@ -1020,60 +963,6 @@
         "enable"
       ]
     },
-    {
-      "id": 43,
-      "type": "FlipStreamSegMask",
-      "pos": [
-        1330.138916015625,
-        291.3784484863281
-      ],
-      "size": [
-        210,
-        66
-      ],
-      "flags": {},
-      "order": 23,
-      "mode": 0,
-      "inputs": [
-        {
-          "name": "tensor",
-          "type": "IMAGE",
-          "link": 84
-        },
-        {
-          "name": "target",
-          "type": "STRING",
-          "link": 86,
-          "widget": {
-            "name": "target"
-          }
-        }
-      ],
-      "outputs": [
-        {
-          "name": "preview",
-          "type": "IMAGE",
-          "links": [
-            85
-          ],
-          "slot_index": 0
-        },
-        {
-          "name": "mask",
-          "type": "MASK",
-          "links": [
-            89
-          ],
-          "slot_index": 1
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "FlipStreamSegMask"
-      },
-      "widgets_values": [
-        ""
-      ]
-    },
     {
       "id": 47,
       "type": "SetLatentNoiseMask",
@@ -1129,7 +1018,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 5,
+      "order": 4,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1178,7 +1067,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 6,
+      "order": 5,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1228,7 +1117,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 7,
+      "order": 6,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1265,7 +1154,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 8,
+      "order": 7,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1355,7 +1244,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 9,
+      "order": 8,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1405,7 +1294,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 10,
+      "order": 9,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1457,7 +1346,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 11,
+      "order": 10,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1494,7 +1383,7 @@
       "flags": {
         "collapsed": true
       },
-      "order": 12,
+      "order": 11,
       "mode": 0,
       "inputs": [],
       "outputs": [
@@ -1528,6 +1417,122 @@
         10,
         1
       ]
+    },
+    {
+      "id": 43,
+      "type": "FlipStreamSegMask",
+      "pos": [
+        1330.138916015625,
+        291.3784484863281
+      ],
+      "size": [
+        210,
+        66
+      ],
+      "flags": {},
+      "order": 23,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "tensor",
+          "type": "IMAGE",
+          "link": 84
+        },
+        {
+          "name": "target",
+          "type": "STRING",
+          "link": 86,
+          "widget": {
+            "name": "target"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "preview",
+          "type": "IMAGE",
+          "links": [
+            85
+          ],
+          "slot_index": 0
+        },
+        {
+          "name": "mask",
+          "type": "MASK",
+          "links": [
+            89
+          ],
+          "slot_index": 1
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "FlipStreamSegMask"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 33,
+      "type": "FlipStreamFilmVfi",
+      "pos": [
+        2270.01904296875,
+        189.28005981445312
+      ],
+      "size": [
+        210,
+        82
+      ],
+      "flags": {},
+      "order": 29,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "frames",
+          "type": "IMAGE",
+          "link": 43
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            44
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "FlipStreamFilmVfi"
+      },
+      "widgets_values": [
+        8,
+        100
+      ]
+    },
+    {
+      "id": 30,
+      "type": "Note",
+      "pos": [
+        -243.73464965820312,
+        21.333702087402344
+      ],
+      "size": [
+        325.1155700683594,
+        135.01312255859375
+      ],
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "properties": {},
+      "widgets_values": [
+        "This workflow depends on additional custom nodes:\nLoRA Tag Loader for ComfyUI\nComfyUI-Florence2\nComfyUI-Frame-Interpolation\n\nUsing lora files:\ndmd2_sdxl_4step_lora_fp16.safetensors from https://huggingface.co/tianweiy/DMD2/tree/main"
+      ],
+      "color": "#432",
+      "bgcolor": "#653"
     }
   ],
   "links": [
@@ -1888,17 +1893,21 @@
   "config": {},
   "extra": {
     "ds": {
-      "scale": 0.7513148009015781,
+      "scale": 0.5993439296133066,
       "offset": [
-        -851.6789375502141,
-        -6.973656498122496
+        262.0605205864992,
+        207.24436173360732
       ]
     },
     "node_versions": {
-      "comfyui_flipstreamviewer": "5a8f1af2e82d99fe545dbabbb494647af0b19bbb",
-      "comfy-core": "0.3.13",
+      "ComfyUI_FlipStreamViewer": "c727b903bb52c60932d2cd2b3841c6248d25854c",
+      "comfy-core": "0.3.14",
       "comfyui_lora_tag_loader": "c1e1a061d42384e06ff6c547c4a00aef7cc96e59"
-    }
+    },
+    "VHS_latentpreview": false,
+    "VHS_latentpreviewrate": 0,
+    "VHS_MetadataImage": true,
+    "VHS_KeepIntermediate": true
   },
   "version": 0.4
 }