ComfyStereo

Introduction

A port of the stereoscopic script used in thygate/stable-diffusion-webui-depthmap-script Most of the credit goes to them, and most of the rest to some LLMs.

https://github.com/user-attachments/assets/b76e6243-557b-454b-9baa-1aac2a7eb22a

Added some new functionality such as edge and direction aware blur of the depth map. I've also managed to add some additional interpolation and stereo distortion methods. By 'I', I mean I tortured ChatGPT for hours.

Things to note

To activate the adaptive blurring, set 'depth_blur_sigma' (the amount of blur applied) to a value greater than 0.

The mask output is imperfect and only provides useful output for the 'No fill' options.

Stereo Image Generation Methods

Summary Table

Method	Description
'No fill'	Simple depth shift with no gap filling.
'No fill - Reverse projection'	Works backward to assign pixel values but leaves gaps.
'Imperfect fill - Hybrid Edge'	Mixes 'Polylines' and 'Reverse projection' for better results.
'Fill - Naive'	Fills gaps with nearest pixel, causing stretching.
'Fill - Naive interpolating'	Uses interpolation to smooth out gaps.
'Fill - Polylines Soft'	Uses polylines with soft edges to maintain structure.
'Fill - Polylines Sharp'	Like 'soft' but with sharper transitions.
'Fill - Post-fill'	'No fill' with edge-aware interpolation and blending.
'Fill - Reverse projection with Post-fill'	'Reverse projection' with directional interpolation and blurring.
'Fill - Hybrid Edge with fill'	Enhanced version of 'Hybrid Edge' with adaptive smoothing.

1. 'No fill'

   • Basic method that applies the depth-based shift but does not fill in gaps left by the transformation.
   • Results in visible holes where pixels are moved but no data is available to replace them.

2. 'No fill - Reverse projection'

   • Instead of shifting pixels away from their original positions, it works backward:
     • For each output pixel, it looks at where it would have originated and assigns a value accordingly.
   • More accurate in some cases but tends to leave gaps.

3. 'Imperfect fill - Hybrid Edge'

   • Combines 'Polylines' and 'Reverse projection' techniques:
     • Uses 'Polylines' for continuous regions to preserve structure.
     • Uses 'Reverse projection' for discontinuous areas to avoid stretching.
   • Works well in most cases but may still struggle with extreme depth changes.

4. 'Fill - Naive'

   • Moves pixels according to depth but fills in gaps using the nearest available pixel.
   • This leads to stretched artifacts but removes the most obvious missing data.

5. 'Fill - Naive interpolating'

   • Like 'naive' but interpolates between neighboring pixels to fill gaps smoothly.
   • Reduces stretching artifacts but still relies on simple heuristics.

6. 'Fill - Polylines Soft'

   • Treats each row of pixels as a polyline and shifts it while attempting to maintain structure.
   • Uses soft edges, meaning transition areas are blended more smoothly.
   • Better at preserving fine details and gradients.

7. 'Fill - Polylines Sharp'

   • Similar to 'Polylines Soft' but with sharper transitions at edges.
   • Avoids excessive blending but may introduce jagged artifacts in areas with sharp depth changes.

8. 'Fill - Post-fill'

   • Applies the 'No fill' method first, then post-processes using:
     • Edge-aware interpolation, attempting to fill gaps based on nearby structures.
     • Blending techniques to smooth out harsh transitions.
   • Improves upon 'No fill' but still has artifacts in extreme cases.

9. 'Fill - Reverse projection with Post-fill'

   • Applies the 'inverse' method first, then post-processes using:
     • Directional interpolation to fill missing areas more intelligently.
     • Blurring corrections to reduce sharp edge artifacts.
   • A more refined version of 'Reverse projection' that significantly improves final image quality.

10. 'Fill - Hybrid Edge with fill'

• An advanced version of 'hybrid_edge' with:
  • Stronger edge-aware processing to ensure smooth depth transitions.
  • Adaptive interpolation based on local depth complexity.
  • Directional-aware smoothing to further refine edges.
• Better in some cases.

---

Installation

Easy method

Use ComfyUI-Manager

Manual install

To install the these nodes, clone this repository and add it to custom_nodes folder in your ComfyUI nodes directory:

git clone https://github.com/Dobidop/ComfyStereo.git
pip install -r requirements.txt

Example workflow

Image

Video